diff --git a/changes/n03-skill-cloud/.spec-superflow.yaml b/changes/n03-skill-cloud/.spec-superflow.yaml new file mode 100644 index 0000000..37096d7 --- /dev/null +++ b/changes/n03-skill-cloud/.spec-superflow.yaml @@ -0,0 +1,47 @@ +version: 1 +state: executing +workflow: full +change_name: n03-skill-cloud +artifact_language: zh-CN +dp_0_confirmed: true +dp_0_result: confirmed +dp_0_timestamp: 2026-09-14T07:30:33Z +dp_0_decisions: | + 变更名称：n03-skill-cloud；一句话意图：基于 ESCO + O*NET 开源词库提供行业技能词云、个人 TOP10 技能匹配度和学习路径接口。 + 工作流：full；选择原因：新增模块、Flyway 数据结构、三个 API、权限与测试，属于跨模块行为变更。 + 约束：仅登录 STUDENT 可访问；行业和职位必填、技能分类默认全部；热度使用 O*NET Importance；无权重技能保留且排后；不调用 AI；不接课程平台；首版仅覆盖互联网、金融、教育及代表性职位；词库通过 Flyway 落库。 + 相关优化：保持聚焦，不纳入平台岗位热度聚合、后台词库维护、学习进度记录或前端开发。 + 沟通方式：按决策点审查，规划产物一次生成后统一确认。 + artifact_language=zh-CN +dp_1_result: confirmed +dp_1_timestamp: 2026-09-14T07:30:33Z +dp_2_result: approved: proposal, three delta specs, design, and tasks validated +dp_2_confirmed: true +dp_2_timestamp: 2026-09-14T07:51:20Z +contract_hash: 819762ae94f2cb3d11820904d013947c367e4bbb32fe8773a3510a253be9ca97 +dp_3_result: approved: execution contract B1-B3; revision 2 allows ESCO primary plus O*NET software_skills supplemental source with machine-traceable importance provenance +dp_3_confirmed: true +dp_3_timestamp: 2026-09-14T07:56:28Z +execution_mode: sdd +plan_revision: 1 +dp_4_result: approved: SDD selected; W1 serial T1-T3, W2 serial T4-T7 depends W1, W3 serial T8-T10 depends W2 +dp_4_confirmed: true +dp_4_timestamp: 2026-09-14T08:00:29Z +batches_completed: 3 +dp_5_result: adjudicated: approved one focused repair for exact per-skill source and full proxy assertions +dp_5_timestamp: 2026-09-14T09:21:01Z +execution_paused: false +verification_blocked: false +verification_outcome: conditional-pass-with-formal-waiver +dp_6_result: pass: 74 Skill Cloud tests, OpenAPI 129 paths, package and startup fix verified; 3 unrelated pre-existing full-suite failures formally waived by user +dp_6_timestamp: 2026-09-15T08:40:05Z +test_result: pass: scoped verification with formal waiver +paused_at: 2026-09-14T09:44:45Z +resumed_at: 2026-09-15T01:18:50Z +resume_count: 1 +paused_reason: user requested immediate pause; W2 may only resume at T4 on manual request +resume_from: T4 +resume_mode: manual +artifacts: + order: [proposal, specs, design, tasks] + skip: [] \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/execution-plan.yaml b/changes/n03-skill-cloud/.superpowers/sdd/execution-plan.yaml new file mode 100644 index 0000000..0495356 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/execution-plan.yaml @@ -0,0 +1,29 @@ +plan_revision: 1 +selected_mode: sdd +contract_hash: 819762ae94f2cb3d11820904d013947c367e4bbb32fe8773a3510a253be9ca97 +created_at: 2026-09-14T08:00:29Z +paused: false +resumed_at: 2026-09-15T01:18:50Z +current: verification +waves: + - id: W1 + strategy: serial + tasks: [T1, T2, T3] + depends_on: [] + state: passed + receipt: pass (T1, T2, T3 final review) + - id: W2 + strategy: serial + tasks: [T4, T5, T6, T7] + depends_on: [W1] + state: passed + receipt: pass (W2-wave; 50 tests) + - id: W3 + strategy: serial + tasks: [T8, T9, T10] + depends_on: [W2] + state: passed_with_external_blocker + receipt: pass (W3 final review; 73 targeted tests, OpenAPI 129 paths) +verification_blocked: false +verification_outcome: conditional-pass-with-formal-waiver +closure_requires: DP-7 confirmation pending \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/progress.md b/changes/n03-skill-cloud/.superpowers/sdd/progress.md new file mode 100644 index 0000000..266f2cd --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/progress.md @@ -0,0 +1,32 @@ +# N03 SDD Progress Ledger + +- 2026-09-14T08:00:29Z Plan revision 1 created for mode SDD. +- W1 eligible: T1 → T2 → T3 (serial, no prerequisite). +- W2 blocked on W1: T4 → T5 → T6 → T7. +- W3 blocked on W2: T8 → T9 → T10.- 2026-09-14T08:05:32Z Task T1: complete (commit b45538d, review W1-T1 PASS, 10 representative positions fixed). +- 2026-09-14T08:19:38Z Task T2: complete (review W1-T2 PASS, RED->GREEN, 2 tests pass). +- 2026-09-14T08:28:32Z Task T2: complete (commit 93477bd, review W1-T2 PASS, migration tests 2/2). +- 2026-09-14T08:42:56Z Task T3: failed attempt 1/3 - critical source-model conflict; review W1-T3 FAIL. Hybrid O*NET software skills not authorized by current contract. +- 2026-09-14T09:07:23Z Task T3: failed attempt 2/3 - review W1-T3-rereview FAIL; data correct but tests only sampled provenance. +- 2026-09-14T09:17:50Z Task T3: failed attempt 3/3 - review W1-T3-rereview-2 FAIL; source data correct but test falsifiability still incomplete. Automatic dispatch stopped; human adjudication required. +- 2026-09-14T09:21:01Z DP-5 adjudication: continue with one focused test-falsifiability repair. +- 2026-09-14T09:38:16Z Task T3: complete after adjudicated repair (final review PASS; 10 tests green; 221 exact mappings; W1 complete). +- 2026-09-14T09:44:38Z User paused execution immediately. Active T4 agent was terminated before completion; partial uncommitted T4 files were removed. W1 remains passed. Resume manually from T4 only; do not start T5-T10 earlier. +- 2026-09-15T01:18:50Z Manual resume accepted. W1 remains passed; W2/T4 is eligible. T5-T10 remain blocked until predecessor reviews pass. +- 2026-09-15T01:29:44Z Task T4: failed attempt 1/3 - review W2-T4 FAIL; category join lacks same-occupation guard and test cannot falsify cross-position mismatch. +- 2026-09-15T01:37:17Z Task T4: complete (commits 0c77824, 52751e5; review W2-T4-rereview PASS; 16 tests). T5 eligible. +- 2026-09-15T01:48:56Z Task T5: failed attempt 1/3 - review W2-T5 FAIL; SkillCloudItem omitted sourceType/sourceCode. +- 2026-09-15T01:54:38Z Task T5: complete (commits e3eecfe, 1666162; review W2-T5-rereview PASS; 24 tests). T6 eligible. +- 2026-09-15T02:08:59Z Task T6: failed attempt 1/3 - review W2-T6 FAIL; tests do not isolate nameZh and alias matching branches. +- 2026-09-15T02:14:37Z Task T6: complete (commit 6eb30e7; review W2-T6-rereview PASS; 40 tests). T7 eligible. +- 2026-09-15T02:23:08Z Task T7: failed attempt 1/3 - review W2-T7 FAIL; missing-skill sequence test not falsifiable because names/IDs were already ascending. +- 2026-09-15T02:36:53Z W2 wave review PASS; 50 tests; W3/T8 eligible. Minor items carried to T8/T9. +- 2026-09-15T02:47:27Z Task T8: failed attempt 1/3 - review W3-T8 FAIL; OpenAPI examples inconsistent and role tests mocked ProfileService. +- 2026-09-15T02:59:54Z Task T8: complete (commit 3e353bd; review W3-T8-rereview PASS; 66 tests). T9 eligible. +- 2026-09-15T03:17:22Z Task T9: complete (commit 7873a42; review W3-T9 PASS; 71 targeted tests). External full-suite blocker recorded: 2 failures + 9 errors in unrelated CareerProgress/JobReview/LocalResume tests; no T9 impact. T10 docs may proceed, final verification remains blocked. +- 2026-09-15T03:29:18Z W3 wave review FAIL: duplicate operationId getMatch and 401 response contract mismatch; full-suite external blocker remains open. +- 2026-09-15T03:40:12Z W3 rereview FAIL: N-I1 existing cancel operationIds changed by global uniqueness check; restore legacy and scope check to new operations. +- 2026-09-15T03:46:09Z W3 final review PASS for T8-T10; 73 targeted tests and OpenAPI/package checks pass. W3 code complete. Final closure blocked by unrelated full-suite failures: 2 failures + 9 errors. Waiver or out-of-scope repair authorization required. +- 2026-09-15T05:48:03Z Runtime startup bug: Hibernate schema validation failed on occupation_skills.importance_is_proxy (found BIT, expected TINYINT). Root cause: forced @JdbcTypeCode(SqlTypes.TINYINT) unlike existing Boolean mappings. Fix dispatched with mapping regression test. +- 2026-09-15T06:03:54Z Runtime bug fix attempt 2: first fix removed forced TINYINT but H2 still stores TINYINT and expects BOOLEAN; 34 tests failed. New hypothesis: add V37 to normalize column to BOOLEAN (MySQL alias TINYINT(1)) and remove test-only preferred type override. +- 2026-09-15T06:20:09Z Runtime startup fix PASS: V37 normalizes importance_is_proxy to cross-dialect BOOLEAN, entity uses default Boolean mapping; 74 targeted tests and package pass without extra JVM properties. diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T1.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T1.md new file mode 100644 index 0000000..f76e17e --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T1.md @@ -0,0 +1,54 @@ +# W1-T1 独立评审报告 + +## 结论 + +**PASS** + +T1 文档交付满足已批准执行契约中 B1 对来源、覆盖边界、映射责任、导入规则和迁移安全的要求，未发现会阻断 T2 开始的 Critical 或 Important finding。 + +## 评审范围与证据 + +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\data\source-manifest.md` +- `D:\1\zhi\zhilai\docs\skill-taxonomy-sources.md` +- 对照依据：`execution-contract.md`、`tasks.md`、`design.md` 及三份 spec。 + +独立检查结果： + +1. 两份文件均存在且非空：manifest 为 8332 bytes/77 行，docs 为 8485 bytes/79 行。移除各自标题及 `docs` 的引言段后，事实正文逐行完全一致。 +2. 固定来源完整：两份文件均明确 ESCO dataset v1.2.1（Last update 10/12/2025，Release v5.7.0）、O*NET 31.0 Database、对应许可、官方归属 URL 和 ESCO 指定署名语句。证据：manifest 第 7-10、47-52 行；docs 第 9-12、49-54 行。 +3. 运行时离线约束明确：禁止运行时访问 ESCO、O*NET 或在线词库，首版数据随 Flyway 进入本地数据库。证据：manifest 第 10、43 行；docs 第 12、45 行。 +4. 项目映射责任明确：项目维护中文名称、别名、细粒度技能分类及行业/职位映射；上游来源 ID 原样保存、不得重编号或伪造。证据：manifest 第 36-40、58-63 行；docs 第 38-42、60-65 行。 +5. T2/T3 分工明确：T2 只建七张表、外键、唯一约束和索引；T3 才导入三类行业、代表性职位、分类、技能、别名及 O*NET Importance。证据：manifest 第 34 行；docs 第 36 行。 +6. 中文名与缺权规则明确：入选技能必须有非空 `name_zh`；无 O*NET Importance 的技能必须保留并写 `importance=null`，查询映射为 `heat=null` 且排在有权重技能之后。证据：manifest 第 37-38 行；docs 第 39-40 行。 +7. 迁移安全规则明确：计划使用 V35/V36；若编号被占用必须回退刷新；不得改写 V1-V34 或任何已执行迁移。当前迁移目录仍以 V34 为最新，V35/V36 均不存在，因此该规则与当前仓库状态一致。证据：manifest 第 42 行；docs 第 44 行。 +8. 覆盖矩阵覆盖互联网、金融、教育三类行业，共 10 个实际列出的代表性职位，计数为互联网 4、金融 3、教育 3，每个职位均给出首批技能分类范围。证据：manifest 第 14-30 行；docs 第 16-32 行。 +9. 未发现 `TBD`、`TODO`、“稍后确定”、“待定”或 placeholder；未发现与已批准 spec、design 或 execution contract 的行为冲突。 +10. 未发现 T1 越界实现：`com.mydreamjob.skill` 主代码和测试目录均不存在，V35/V36 迁移也不存在，符合 T1 只交付来源文档的范围。 + +## Findings + +### Critical + +无。 + +### Important + +无。 + +### Minor + +1. **派发文字中的“11 个职位”与实际枚举的 10 个职位不一致。** + 已批准 execution-contract、tasks.md、design.md 和 spec 均只规定“互联网、金融、教育及代表性职位”，没有规定硬性 11 个；本轮派发提示中的枚举实际为互联网 4、金融 3、教育 3，共 10 个。两份交付物覆盖全部实际枚举职位，并明确说明未自行新增未批准职位。该差异不影响 T1 验收，也不阻断 T2；建议后续派发统一按 10 个职位表述，除非人工补充第 11 个已批准职位名称。 + +2. **T1 固化的是技能分类范围和技能级来源规则，而不是逐个技能实例的 ESCO/O*NET 来源 ID 与中文标准名清单。** + 这与 T1 的“可审查数据边界”目标一致，不构成当前阻断；具体技能级来源 ID、中文标准名及 Importance 映射仍须在 T3 种子数据中逐项落实，并由迁移测试验证。 + +## 未决问题 + +- 无阻断 T2 的未决问题。 +- T3 必须履行文档已固定的技能级约束：稳定来源 ID 原样保存、`name_zh` 非空、缺失 Importance 写 `null`。 +- 如后续验收仍坚持“11 个职位”，必须由人工提供第 11 个已批准职位名称；当前无任何已批准依据可推导该职位。 + +## 阻断项 + +无。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T2.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T2.md new file mode 100644 index 0000000..50a6378 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T2.md @@ -0,0 +1,98 @@ +# W1-T2 独立评审报告 + +## 状态 + +完成。本次评审未修改任何实现文件，仅写入本报告。 + +## Verdict + +**PASS** + +T2 已建立执行契约要求的七张词库表、关键字段、外键、唯一约束、CHECK 约束和两个查询索引；迁移可执行，且未顺带实现 T3 种子数据或 Java 业务代码。未发现阻断 T3 的 Critical 或 Important finding。 + +## 评审对象与依据 + +评审对象： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\resources\db\migration\V35__skill_cloud_taxonomy.sql` +- `D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillTaxonomyMigrationTest.java` + +对照依据： + +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\execution-contract.md` +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\design.md` +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\tasks.md` +- T2 RED/GREEN 日志 +- `git diff --cached`、`git diff HEAD` 和 `git diff --check` +- JDK 17 下的独立测试执行结果 + +## 独立检查结果 + +1. **七张表及关键列存在。** V35 第 5、17、28、44、60、77、88 行分别创建 `skill_source_versions`、`skill_industries`、`skill_occupations`、`skill_categories`、`skills`、`skill_aliases`、`occupation_skills`。测试通过真实 `DatabaseMetaData` 查询七个表的全部预期列，见测试第 55-74、223-240 行，不是 SQL 文本断言。 + +2. **外键真实存在。** V35 第 40-41、57、74、85、105-108 行建立行业、来源版本、职位、分类、技能和别名之间的 9 条外键；测试第 86-94 行通过 `getImportedKeys` 逐条验证。 + +3. **唯一约束真实存在。** V35 第 14、25、39、53、71-72、83-84、98 行覆盖来源版本、行业编码、行业内职位编码、职位内分类编码、技能编码、技能来源、别名归一化值、技能别名和职位技能关系；测试第 76-84 行逐项验证唯一索引。 + +4. **CHECK 约束符合契约。** 分类模板限定为 `PROGRAMMING_LANGUAGE`、`FRAMEWORK`、`TOOL`、`GENERAL`，见 V35 第 54-56 行；技能来源限定为 `ESCO`、`O*NET`、`PROJECT`，见第 73 行；关系类型限定为 `ESSENTIAL`、`OPTIONAL`、`PROJECT`，见第 102-104 行。非法模板、非法来源和非法关系类型均通过实际失败插入验证，见测试第 159-169、182-186 行。 + +5. **`importance` 可空且无默认权重。** V35 第 93 行定义 `DECIMAL(4,2)` 且没有 `NOT NULL` 或 `DEFAULT`；第 99-101 行只约束为空或位于 0.00-5.00。测试插入时省略该列并断言查询结果为 `null`，随后验证 `4.25` 可保存，见测试第 133-153 行。 + +6. **中文名称非空。** 行业、职位、分类和技能的中文名称均定义为 `NOT NULL`，见 V35 第 20、32、48、63 行；测试第 176-181 行通过插入 `name_zh=null` 并断言失败。 + +7. **两个查询索引真实存在。** V35 第 109-110 行建立 `(occupation_id, category_id)` 与 `(occupation_id, importance)` 索引；测试第 96-97 行通过数据库元数据验证。`occupation_id + importance` 可被 MySQL 正向或反向扫描用于热度降序查询。 + +8. **迁移范围正确。** `git diff --name-status HEAD` 在迁移和主代码范围内仅显示新增 `V35__skill_cloud_taxonomy.sql` 及新测试文件；V1-V34 未修改，V36 尚不存在，`src/main/java` 无变更，未顺带实现 T3 或 Java 业务代码。 + +9. **测试失败有效性、隔离和清理。** RED 日志第 60-125 行显示 V35 缺失时两项测试分别因缺表失败；测试使用独立的 `integrationdb_skill_taxonomy` H2 数据库，见第 32-33 行；`@AfterEach` 按外键逆序清理插入数据，见第 196-221 行。未发现仅源码文本断言、共享默认 H2 库污染或明显漏清理。 + +10. **MySQL/H2 兼容性。** `AUTO_INCREMENT`、InnoDB、`utf8mb4_unicode_ci`、`DATETIME DEFAULT/ON UPDATE CURRENT_TIMESTAMP`、表内 `KEY`、CHECK 和命名外键均属于 MySQL 8.4 支持的语法；H2 在 `MODE=MySQL` 下已成功执行整份迁移。未发现仅 H2 成立的 DDL 结构。由于本机没有 Docker，且本地 MySQL 服务无可用测试凭据，本报告未进行真实 MySQL 样库执行；该验证限制不阻断仅要求 H2 迁移测试的 T2。 + +## 独立测试结果 + +执行环境与命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' -Dtest=SkillTaxonomyMigrationTest test +``` + +结果： + +```text +Java version: 17.0.16 +Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 36.770 s +``` + +提供的 GREEN 日志与独立执行结果一致：JDK 17.0.16、2 项测试全部通过、`BUILD SUCCESS`。RED 日志也证明测试确实依赖真实 V35 迁移结果。 + +## Findings + +### Critical + +无。 + +### Important + +无。 + +### Minor + +1. **迁移测试只断言列存在，不直接断言全部列类型和可空性。** 测试第 223-240 行收集的是列名；当前 DDL 的类型和可空性经人工核对符合 design/契约，因此不阻断，但未来类型回归不会全部被测试发现。建议后续在数据库元数据断言中补充 `importance DECIMAL(4,2) nullable`、各 `name_zh NOT NULL` 及关键 ID 列类型检查。 + +2. **`importance` 的越界分支未行为测试。** 测试覆盖了 `null` 和 `4.25`，但未尝试 `-0.01` 或 `5.01`，因此当前 CHECK 上下界只经人工核对，见 V35 第 99-101 行。风险较低，可在 T3 或端到端回归中补充。 + +3. **约束失败断言捕获所有 `DataAccessException`，未锁定具体约束。** 测试第 330-332 行通过这一点；当前各失败用例的其余字段均合法，因此实际原因基本可确定，但测试信息量和诊断精度略低。建议后续断言约束名或 SQL 状态/错误码，进一步防止因无关错误而误通过。 + +4. **测试文件末尾存在多余空白行，`git diff --check` 报告 `new blank line at EOF`。** 证据为测试文件第 334 行。纯格式问题，不阻断 T3。 + +## 未决问题 + +- 本报告仅在 H2 `MODE=MySQL` 下实际执行了迁移；真实 MySQL 8.4 执行应在已有 MySQL 凭据或 Docker 环境中由后续发布级验证补充。静态检查未发现 MySQL 不兼容语法。 +- 分类必须属于职位属于服务层校验责任，T2 只提供基础外键；该跨列一致性须在 T4/T8 按 design 第 117、170 行落实并验证。 + +## 阻断项 + +无。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-final-review.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-final-review.md new file mode 100644 index 0000000..f1db1af --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-final-review.md @@ -0,0 +1,136 @@ +# W1-T3 最终独立复审报告（DP-5 唯一复审） + +## 状态 + +完成。本轮只读取实现、V35/V36、期望映射、证据快照和前序评审报告，并独立执行指定测试、CSV 来源审计与文件哈希核验；未修改任何实现或测试文件，仅写入本报告。 + +## Verdict + +**PASS** + +第三轮 findings 已全部关闭，允许进入 T4。未发现 Critical 或 Important 阻断项。 + +## 独立测试输出 + +环境： + +- `JAVA_HOME=D:\java\jdk` +- Java：OpenJDK 17.0.16 +- Maven：`D:\maven\apache-maven-3.9.7\bin\mvn.cmd` +- 工作目录：`D:\1\zhi\zhilai\mydreamjob-server` + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest' test +``` + +结果： + +```text +SkillCloudSeedDataIntegrationTest: Tests run: 8, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyMigrationTest: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 +Total: Tests run: 10, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 39.479 s +``` + +## 第三轮 Finding 逐项关闭状态 + +### 1. 47/47 条 O*NET 技能来源逐技能精确锁定：关闭 + +- `skill-cloud-expected-mappings.csv` 含 47 条 `SKILL_SOURCE` + `O*NET` 记录，47 个技能编码和 47 个来源三元组均唯一。 +- `verifyExpectedMappings` 先通过 `assertSameMappingKeys` 精确比较期望文件与数据库的全部实体键；随后对每一条执行 `assertMappingMatches`，逐条比较 `source_type` 和 `source_code`。 +- `verifySkillEvidence` 逐条将 47 个 `skill_code -> SOC|Workplace Example|Element ID` 与证据快照核对，并用 `assertEvidenceKeySet` 要求期望软件来源集合与快照 47 条集合完全相等。 +- 交换任意两个 O*NET 技能来源时，至少一行会在逐字段断言处失败，不再存在仅验证“来源三元组集合合法”的缺口。 +- `assertMappingVerifierRejectsTampering` 的交换来源自检通过，且它调用的是全量循环实际使用的同一个比较函数。 + +### 2. 137/137 条非空 Importance 全字段精确断言：关闭 + +- 期望文件包含 139 条职位技能关系，其中 137 条非空 Importance、2 条空 Importance。 +- 137 条非空关系均包含非空 `expected_source_code`、`expected_element_id`、`expected_importance`，且 `expected_is_proxy=1`。 +- 全量循环对每条关系精确比较 `source_type`、`source_code`、`element_id`、`importance` 和 `importance_is_proxy`；数值使用 `0.001` 容差，其余字段使用严格相等。 +- 任意一条非空关系的 proxy 从 `1` 改为 `0`、来源 SOC/Element ID 被替换、或数值被修改，都会在对应关系上失败。 +- 2 条空 Importance 关系均断言来源 SOC、Element ID、Importance 为空且 `importance_is_proxy=0`。 + +### 3. expected-mappings 覆盖且仅覆盖 10/10 occupation、72/72 skill、139/139 relation：关闭 + +- 期望文件共 221 条数据行：`OCCUPATION_SOURCE=10`、`SKILL_SOURCE=72`、`RELATION=139`。 +- `record_type + entity_key` 重复数：0。 +- `assertExpectedMappingCounts` 锁定三类记录数量；`assertSameMappingKeys` 进一步比较期望与数据库完整键集合，缺项和多余项都会失败。 +- 独立测试通过证明 Flyway 实际生成的 10 个职位、72 个技能和 139 条关系与期望文件一一对应。 + +### 4. expected-mappings 与证据快照交叉一致，且 reference 匹配官方 O*NET 31.0 CSV：关闭 + +独立解析结果： + +| 检查项 | 结果 | +| --- | ---: | +| `onet-31-reference.csv` 数据行 | 92 | +| SOFTWARE 证据 | 47，键无重复 | +| IMPORTANCE 唯一证据 | 45，键无重复 | +| 47/47 SOFTWARE 在官方 `software_skills.csv` 精确命中 | 通过 | +| 45/45 IMPORTANCE 在声明官方 CSV 中精确命中 SOC、Element ID、Scale ID=IM、Data Value | 通过 | +| 137/137 加权关系的证据键和值可回溯到 reference | 通过 | +| 期望文件引用的 SOFTWARE 与 reference SOFTWARE 集合相等 | 通过 | +| 期望文件引用的唯一 IMPORTANCE 键与 reference IMPORTANCE 集合相等 | 通过 | + +`target/onet-31` 当前 CSV 哈希： + +```text +software_skills.csv 6AABB96B464288DB849580E6510530EFFF3F80E25DDE2BB23F1FC36BB526B016 +work_activities.csv 6C266205B2926C0C854A7A577AAE29FE492C71BB0DF7E197ADCA830B58FE4EA1 +knowledge.csv 88092E17D16096CDEBE292FEE4389DEB460EDDA24AE8B4637212C69C71562F55 +essential_skills.csv 68C4D31D8C55077238395DCA40CC4E6D32E6A608B5474BB8264B1F98AE415BD7 +transferable_skills.csv F17D0419E818A3BF2F55076510C76C4AEF01CC8F5AEE2670D2F44F6F44AC60F7 +occupation_data.csv A09EAE1D6609686E44E05B7290993A1C8B523D8CA224BC0EEDC194C855C3EE02 +``` + +这些 CSV 哈希与前次独立审计记录的 O*NET 31.0 官方发布包解压文件哈希一致。本轮再次请求官方 ZIP 时收到 HTTP 200，但连接在完成前中断；因此未重复完成 ZIP 级哈希计算。该限制不影响本轮映射验证，因为 reference 已与哈希不变的官方 CSV 逐行精确匹配。 + +### 5. 测试独立性、无 target 依赖、可证伪性：关闭 + +- `SkillCloudSeedDataIntegrationTest` 只使用 `ClassPathResource` 读取 `skill-cloud/onet-31-reference.csv` 和 `skill-cloud/skill-cloud-expected-mappings.csv`。 +- 测试源码未引用 `V36` 文件、`target/onet-31`、文件系统绝对路径或其他运行后才生成的期望数据。 +- 期望映射是独立静态资源；数据库实际值通过 Flyway 迁移和 JDBC 读取，不是从 V36 文本反推后生成期望文件。 +- 测试可在没有 `target/onet-31` 的前提下运行；`target/onet-31` 仅在本次人工来源审计中用于交叉核对。 +- 自检覆盖来源交换、proxy 翻转和 Importance 篡改，且生产断言路径已对全部 221 条期望实体执行。 + +### 6. 68/68 分类、8 个唯一 ESCO occupation URI、混合来源契约及 V35/V36 约束：关闭 + +- 测试 `seedDataCoversIndustriesOccupationsAndEveryDeclaredCategory` 通过：3 个行业、10 个职位、68 个分类，10 个职位的分类清单逐项一致，每个分类至少关联 1 条技能关系，每个职位至少 10 个加权技能。 +- 10 个职位均使用 ESCO 职位来源；8 个 URI 值唯一，均符合 `http://data.europa.eu/esco/occupation/...` 格式。DevOps 与量化开发按已批准契约复用软件开发职位 URI。 +- 混合来源契约保持明确：25 个技能为 ESCO，47 个具体技术为 O*NET `software_skills` 补充；`skills.source_type`、来源三元组唯一约束和来源版本外键一致。 +- V35 仍具备所需唯一约束、外键、查询索引、`importance` 0–5 CHECK、非空 Importance 来源完整性 CHECK 和 proxy `0/1` CHECK。 +- V36 的 137 条非空 Importance 均保存 O*NET SOC、Element ID 和 proxy=1；2 条无可靠映射的关系保留技能并写入 NULL，proxy=0。 +- H2 `MODE=MySQL` 集成迁移测试通过；本轮未新增真实 MySQL 8.4 执行要求。 + +## 文件哈希 + +```text +A9399B4C1D9525F3ED91694FE8F02433C1BF634B458C693D91A4014C1252183B V35__skill_cloud_taxonomy.sql +0D5D39A556B6D72CDB21422ED7020B1EAEC0BE8661635B497D12B40D9D104724 V36__skill_cloud_seed.sql +2A281B099AF4DB5E8117A99D8281191EBC60E68B4BC0E153551AD2013B0071A5 SkillCloudSeedDataIntegrationTest.java +4672287B3197784E4144EE36C322AD27AB3BD6955D6626D885E31349C9A19435 onet-31-reference.csv +F6D7545F01C18A6C13D17C00A051D757AED4A8BD2BEBA34757D5DEC1403A6C7B skill-cloud-expected-mappings.csv +``` + +## Findings + +### Critical + +无。 + +### Important + +无。 + +### Minor + +- 官方发布包 ZIP 本轮重下载未完成，reference 的官方身份依赖“官方 CSV 哈希不变 + 前次已建立的官方 ZIP 审计记录”。当前不影响 T3 正确性或 T4 放行。 +- 测试不会联网重新校验官方 CSV 哈希；这是有意的离线设计，但后续若更新 reference，仍需人工保留来源审计记录。 + +## 是否允许进入 T4 + +**允许。** 第三轮两项 Important finding 均已关闭，10/10 测试通过，未发现新的 Critical 或 Important 问题。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-rereview-2.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-rereview-2.md new file mode 100644 index 0000000..d3b97a9 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-rereview-2.md @@ -0,0 +1,207 @@ +# W1-T3 最终独立复审报告（第二轮修复） + +## 状态 + +完成。本次复审只读取实现、迁移、来源清单、前两轮报告和本地/官方 CSV，并独立执行指定测试、来源审计及 ESCO URI 只读核验；未修改任何实现文件，仅写入本报告。 + +## Verdict + +**FAIL** + +第二轮唯一 Important finding 尚未完全关闭： + +- 47 条 O*NET `software_skills` 来源当前数据全部真实，但测试只验证“每条 DB 三元组存在于快照集合中”，没有验证“每个技能代码对应唯一正确的三元组”；交换两个未被单点断言的技能来源后，测试仍会通过。 +- 137 条非空 Importance 的值均已逐条核对，但测试对 `importance_is_proxy` 仍主要是 `0/1` 枚举检查，仅 10 条做 `proxy=1` 精确断言；将其余 127 条中的任一 `1` 改为 `0` 后，测试仍会通过。 + +以上属于回归保护缺口，不阻断当前种子数据的正确性，但按 TO-12 和本轮“篡改必须失败”的要求，仍阻断 T4。 + +## 独立测试结果 + +环境： + +- `JAVA_HOME=D:\java\jdk` +- Maven：`D:\maven\apache-maven-3.9.7\bin\mvn.cmd` +- 工作目录：`D:\1\zhi\zhilai\mydreamjob-server` + +执行： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest' test +``` + +结果： + +```text +SkillCloudSeedDataIntegrationTest: Tests run: 7, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyMigrationTest: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 +Total: Tests run: 9, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 39.800 s +``` + +## 来源审计方法与结果 + +### 1. O*NET 31.0 官方来源核验 + +独立下载官方发布包： + +- URL：`https://www.onetcenter.org/dl_files/database/db_31_0_csv.zip` +- HTTP：`200` +- 文件大小：`16,237,378` bytes +- HTTP `Last-Modified`：`Tue, 25 Aug 2026 03:46:00 GMT` +- 官方 ZIP SHA-256：`55033FC68B4C13EC23E7F74DC6378660F6E854E75D55D6E333AE0A761D3987CD` + +从官方 ZIP 解压后与 `mydreamjob-server/target/onet-31` 逐文件比较，SHA-256 完全一致： + +| 文件 | SHA-256 | +| --- | --- | +| `software_skills.csv` | `6AABB96B464288DB849580E6510530EFFF3F80E25DDE2BB23F1FC36BB526B016` | +| `work_activities.csv` | `6C266205B2926C0C854A7A577AAE29FE492C71BB0DF7E197ADCA830B58FE4EA1` | +| `knowledge.csv` | `88092E17D16096CDEBE292FEE4389DEB460EDDA24AE8B4637212C69C71562F55` | +| `essential_skills.csv` | `68C4D31D8C55077238395DCA40CC4E6D32E6A608B5474BB8264B1F98AE415BD7` | +| `transferable_skills.csv` | `F17D0419E818A3BF2F55076510C76C4AEF01CC8F5AEE2670D2F44F6F44AC60F7` | +| `occupation_data.csv` | `A09EAE1D6609686E44E05B7290993A1C8B523D8CA224BC0EEDC194C855C3EE02` | + +因此，`target/onet-31` 中的本轮 CSV 来源与官方 O*NET 31.0 发布包一致，不是本地改写版本。 + +### 2. 证据快照与官方 CSV 对照 + +`mydreamjob-server/src/test/resources/skill-cloud/onet-31-reference.csv`： + +- 总行数：92 条数据行。 +- `SOFTWARE`：47 条，47 个唯一键，无重复。 +- `IMPORTANCE`：45 条唯一 `SOC + Element ID` 组合，无重复。 +- 47/47 条 `SOFTWARE` 三元组均精确命中 `software_skills.csv`。 +- 45/45 条 `IMPORTANCE` 均命中声明的官方 CSV，`Data Value` 精确一致。 +- 未发现无依据行、伪造三元组或伪造数值。 +- `SOFTWARE` 的 `data_value` 均为空，符合 `software_skills` 没有 Importance 值的结构。 + +结论：当前证据快照确实来自 O*NET 31.0 官方 CSV，数据与官方 CSV 完全一致。 + +### 3. V36 数据全量审计 + +独立解析 V36 后确认： + +| 项目 | 结果 | +| --- | ---: | +| O*NET 技能 | 47 | +| DB 使用的 SOFTWARE 三元组集合与快照集合相等 | 是 | +| 职位技能关系 | 139 | +| 非空 Importance | 137 | +| 空 Importance | 2 | +| 非空关系正确命中快照键和值 | 137/137 | +| 非空关系当前 `importance_is_proxy=1` | 137/137 | +| 空关系的 SOC/Element ID 为 NULL、proxy=0 | 2/2 | +| 137 条关系使用的唯一 Importance 键 | 45 | +| 快照 IMPORTANCE 键集合与实际使用集合相等 | 是 | + +结论：当前数据层没有发现伪造来源、错误数值或错误代理标记。 + +### 4. Manifest 分类覆盖 + +从 `source-manifest.md` 独立解析 10 个职位及声明分类，并与 V36 对比： + +- 职位：10/10 一致。 +- 分类：68/68 存在。 +- 每个声明分类至少关联 1 条技能关系；本轮独立统计最小值为 2、最大值为 3。 +- 未发现分类遗漏或职位—分类清单不一致。 + +### 5. ESCO 职位 URI + +V36 与测试中的 10 个职位映射一致，其中 8 个唯一 ESCO occupation URI。对 8 个 URI 调用 ESCO 官方只读 API，全部返回 HTTP `200`： + +- 前端开发工程师 URI +- 后端开发工程师 URI +- 测试工程师 URI +- 金融数据分析师 URI +- 风控建模工程师 URI +- 教育产品经理 URI +- 教学设计师 URI +- 教育数据分析师 URI + +DevOps、量化开发复用软件开发者 URI 的项目代理关系已在 V36 注释中明确，和当前批准清单一致。 + +## 上一轮 Finding 关闭状态 + +| 第一轮/第二轮 Finding | 状态 | 证据 | +| --- | --- | --- | +| C-1：混合来源未经批准 | 关闭 | 契约、design、source manifest 均已明确 ESCO 主来源 + O*NET `software_skills` 补充；V35/V36 字段语义与修订契约一致。 | +| I-1：代理 SOC/ESCO URI 不可追溯 | 关闭 | 10 个职位均有 ESCO source version 和 URI；8 个唯一 URI 独立返回 200；代理 SOC、Element ID、proxy 均落表。 | +| I-2：manifest 分类遗漏 | 关闭 | 68/68 分类存在，每个分类至少 1 条关系，清单与 V36 一致。 | +| I-3a：Importance 逐条可追溯数据 | 关闭 | V35 增加来源字段；V36 的 137/137 非空关系均匹配官方快照，当前数据真实。 | +| I-3b：测试全量覆盖 47/47 软件来源 | **部分关闭** | 47 条都做集合成员验证，但只有 5 条做 `skillCode -> sourceCode` 精确映射断言；未断言的两个技能交换来源仍可通过。 | +| I-3c：测试全量覆盖 137/137 非空 Importance | **部分关闭** | 137 条都验证来源键和数值；但 `proxy` 只对 10 条精确断言，其余 127 条可在 1 和 0 之间翻转而不失败。 | + +## Findings + +### Important-1：47 条 O*NET 软件来源未做完整的逐技能精确映射断言 + +位置： + +- `mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudSeedDataIntegrationTest.java:160-183` +- 精确映射单点断言：`185-189` +- 辅助断言：`474-478` + +问题： + +当前循环对每条 DB `source_code` 只检查其三元组是否存在于快照中，再要求 DB 行数与快照行数相等。它验证了“集合完整”，但没有验证每个技能代码必须绑定哪个三元组。 + +可证伪反例： + +- 任意交换两个未被 `assertOnetSkillSource` 覆盖的 O*NET 技能来源，例如 `SPRING_BOOT` 与 `DJANGO`。 +- DB 仍然有 47 个唯一且合法的三元组，集合仍与快照相等。 +- 现有 5 条单点断言不涉及这两个技能，因此测试仍通过，但技能溯源已经被错误交换。 + +影响： + +`skills.source_code` 是来源追踪字段。来源交换会静默破坏“技能到上游来源 ID”的准确性，当前回归测试无法阻止。 + +修复要求： + +- 在快照中增加 `skill_code`（仅 SOFTWARE 行使用），或新增完整的 `skillCode -> SOC|Workplace Example|Element ID` 映射资源。 +- 对 47/47 条 O*NET 技能逐条断言精确来源，而不是仅做集合成员检查。 + +### Important-2：Importance Proxy 标记未做全量语义断言 + +位置： + +- 枚举检查：`SkillCloudSeedDataIntegrationTest.java:201-203` +- 循环内代理检查：`229-230` +- 仅 10 条精确断言：`271-294` +- 精确断言辅助方法：`481-494` + +问题： + +`assertTrue(proxy == 0 || proxy == 1)` 只证明值处于合法范围，不能证明它符合种子数据的来源语义。V36 当前明确要求 137 条非空 Importance 全部是跨来源代理，必须为 `1`；测试只对 10 条做 `proxy=1` 断言。 + +可证伪反例： + +- 将任意一个未被 `assertImportance` 覆盖的非空关系从 `importance_is_proxy=1` 改为 `0`。 +- 该行仍满足数据库 CHECK 和测试的 `0/1` 判断，SQL 计数也不检查非空代理必须为 1，因此测试仍通过。 +- `importance`、SOC、Element ID 和 Data Value 均未发生变化，但来源语义已经被篡改。 + +影响： + +代理标记是来源审计字段。当前测试不能保证 137 条全部保持已批准的 proxy 语义，无法满足“篡改 Proxy 标记必须失败”的要求。 + +修复要求： + +- 明确断言所有非空 Importance 的 `importance_is_proxy=1`，以及所有空 Importance 的 `importance_is_proxy=0`；或为 137 条关系提供逐关系期望映射。 +- 若业务允许直接 O*NET Importance 与代理 Importance 并存，需要按关系保存期望值并逐条断言。 + +### Minor-1：快照的官方来源属性依赖评审时外部核验，未被自动化锁定 + +`loadEvidence` 只验证快照头部、列数、允许的 `source_file` 名称和键唯一性，不会读取官方 CSV 或校验官方发布包哈希。 + +当前快照已由本次复审用官方 ZIP 哈希确认，因此不阻断；但未来若快照的 `source_file` 被改成同一允许集合内的另一个文件，只要测试资源本身同步修改，现有测试无法识别来源文件标签变化。建议在来源文档或测试资源旁记录官方 ZIP/CSV 哈希，作为后续审计基线。 + +## 未决问题 + +- T3 当前数据层已通过，不能因为数据正确就忽略 Important-1 和 Important-2 的回归缺口。 +- 真实 MySQL 8.4 尚未执行；本轮仅按要求执行 H2 `MODE=MySQL`。该限制沿用前轮，不构成本次新的阻断项。 +- V35/V36 当前满足修订后的混合来源契约；未发现需要再次 Rewind 的契约冲突。 + +## 阻断项 + +T3 仍为 **FAIL**。在补齐 47/47 逐技能精确来源映射和 137/137 Importance 代理语义断言前，不应进入 T4。当前测试 9/9 通过只能证明现有数据与当前快照一致，不能证明上述两类篡改必然失败。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-rereview.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-rereview.md new file mode 100644 index 0000000..6dc2ae9 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3-rereview.md @@ -0,0 +1,158 @@ +# W1-T3 独立复审报告（聚焦修复） + +## 状态 + +完成。仅读取实现、契约、来源数据与本地 O*NET CSV，并独立执行测试和来源审计；未修改任何实现文件，仅写入本报告。 + +## Verdict + +**FAIL** + +第一轮 Critical 已关闭，I-1、I-2 的当前数据问题已关闭；I-3 的“逐条可追溯数据”已关闭，但对应的回归测试仍未关闭：当前测试只精确断言 5/47 条 O*NET 技能来源和 10/137 条非空 Importance，且没有任何 CSV 驱动的来源完整性测试。多数不存在或伪造的 O*NET 行、错误 Importance 仍会通过测试。该项属于 Important，阻断 T4。 + +## 评审对象与依据 + +实现文件： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\resources\db\migration\V35__skill_cloud_taxonomy.sql` +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\resources\db\migration\V36__skill_cloud_seed.sql` +- `D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillTaxonomyMigrationTest.java` +- `D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillCloudSeedDataIntegrationTest.java` + +依据： + +- 修订后的 `execution-contract.md`、`design.md`、`tasks.md` +- `changes/n03-skill-cloud/data/source-manifest.md` +- `docs/skill-taxonomy-sources.md` +- 第一轮报告 `changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3.md` +- 修复报告日志 `mydreamjob-server/target/skill-taxonomy-fix-green.log` +- 本地 O*NET 31.0 CSV 与 `changes/n03-skill-cloud/data/esco-skill-lookup.json` +- ESCO 官方只读 API（用于职位 URI 真实性核验） + +## 独立测试结果 + +环境： + +- `JAVA_HOME=D:\java\jdk` +- Maven：`D:\maven\apache-maven-3.9.7\bin\mvn.cmd` +- 工作目录：`D:\1\zhi\zhilai\mydreamjob-server` + +执行： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest' test +``` + +结果： + +```text +SkillCloudSeedDataIntegrationTest: Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyMigrationTest: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 +Total: Tests run: 8, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 40.195 s +``` + +## 第一轮 Finding 关闭状态 + +| 第一轮 Finding | 状态 | 独立证据 | +| --- | --- | --- | +| C-1：混合来源未授权，职位与关系来源被替换 | **关闭** | `execution-contract.md:7,20,56` 已明确授权“ESCO 主来源 + O*NET `software_skills` 补充”；`design.md:32-43` 与 `source-manifest.md:35-43` 同步；V36 中 10 个职位均使用 ESCO occupation URI，25 条 ESCO 技能和 47 条 O*NET 补充技能分源导入。 | +| I-1：金融/风控/教育产品经理代理映射不明确或 URI 不真实 | **关闭** | V36:6-28 明确代理 SOC；V35:93-96 将目标来源与 Importance 来源拆列；V36:274-413 的 137 条非空热度均设置 `importance_is_proxy=1`。独立调用 ESCO 官方 API，10 个职位行对应的 8 个唯一 URI 全部返回 200，标题分别为 web developer、software developer、software tester、financial analyst、risk manager、ICT product manager、instructional designer、data analyst。 | +| I-2：manifest 分类遗漏，测试只要求至少一个分类 | **当前数据关闭** | manifest 10 个职位的分类清单与 V36:59-127 的实际 68 个分类逐项一致，测试也已改为精确比较完整清单；独立解析确认 68/68 分类至少关联 1 条技能。剩余测试弱点见 Minor-1。 | +| I-3：Importance 缺少逐技能来源映射和有效验证 | **部分关闭，仍为 Importance 阻断** | V35:94-110 已增加 SOC、Element ID、代理标记和完整性约束；V36:274-413 的 137/137 条非空热度均带完整字段，并可精确核对 CSV。但测试只覆盖少量样本，不能阻止多数来源或热度被篡改；详见 Important-1。 | + +## 来源审计方法与数量 + +### 1. SQL 结构审计 + +独立解析 V36 的 `INSERT` 行，结果： + +| 对象 | 数量 | +| --- | ---: | +| 来源版本 | 2 | +| 行业 | 3 | +| 职位 | 10 | +| 技能分类 | 68 | +| 技能 | 72（ESCO 25，O*NET 47） | +| 技能别名 | 72 | +| 职位技能关系 | 139 | +| 非空 Importance | 137 | +| 空 Importance | 2 | +| 每个职位非空 Importance 最小值 | 12 | + +V36 仅有 7 条 `INSERT` 语句，没有 `CREATE`、`ALTER`、`DROP`、`UPDATE`、`DELETE`。`com.mydreamjob.skill` 主代码包不存在，未提前实现 T4；V1-V34 没有改动。 + +### 2. ESCO 来源审计 + +- 25/25 条 `source_type='ESCO'` 技能的 URI 都能在 `esco-skill-lookup.json` 中找到。 +- 10/10 个职位行的 `source_version_id=1`，职位来源均为 ESCO occupation URI。 +- 10 个职位使用 8 个唯一 ESCO occupation URI；`BACKEND_DEVELOPER`、`DEVOPS_ENGINEER`、`QUANTITATIVE_DEVELOPER` 复用 software developer URI，V36:47 已明确标注为项目代理。 +- 对 8 个唯一 URI 独立调用 `https://ec.europa.eu/esco/api/resource/occupation?uri=...&language=en`，全部 HTTP 200，返回的 URI 与请求一致。 + +### 3. O*NET `software_skills` 审计 + +- 使用本地 `mydreamjob-server/target/onet-31/software_skills.csv`（31,821 行）。 +- 47/47 条 O*NET 技能的 `(O*NET-SOC Code, Workplace Example, Element ID)` 三元组精确命中 CSV，无缺失或伪造组合。 +- 当前测试对这 47 条来源只有正则格式检查，另精确断言的来源样本为 5 条。 + +### 4. Importance 审计 + +- 使用本地 `work_activities.csv`、`knowledge.csv`、`essential_skills.csv`、`transferable_skills.csv`，共解析出 99,231 个 `(SOC, Element ID, Data Value)` 的 `IM/Importance` 键。 +- 137/137 条非空 Importance 均可按 `(importance_source_code, importance_element_id, importance)` 精确命中至少一个 CSV。 +- 137/137 条均满足 `importance_source_code` 和 `importance_element_id` 非空，且 `importance_is_proxy=1`。 +- 2/2 条空 Importance 的 SOC、Element ID 均为 NULL，`importance_is_proxy=0`。 +- 当前测试对 137 条非空热度中的 10 条做精确值断言，其余 127 条只检查字段存在，不检查数值是否真实。 + +## 阻断项 + +### Important-1：回归测试不能覆盖大多数 O*NET 来源与 Importance 篡改 + +**位置** + +- `SkillCloudSeedDataIntegrationTest.java:132-139`：47 条 O*NET 来源只验证正则格式。 +- `SkillCloudSeedDataIntegrationTest.java:141-145`：仅精确断言 5 条 O*NET `software_skills` 行。 +- `SkillCloudSeedDataIntegrationTest.java:152-163`：只验证非空字段是否存在及来源版本名称是否为 O*NET。 +- `SkillCloudSeedDataIntegrationTest.java:165-187`：仅精确断言 10 条 Importance 及 2 条 NULL。 + +**独立量化** + +- O*NET 技能总数为 47，精确来源断言为 5，剩余 42 条不存在“CSV 行必须存在”的回归保护。 +- 非空 Importance 总数为 137，精确值断言为 10，剩余 127 条即使被改成任意 0-5 的数值，现有测试仍会通过。 +- 测试文件未读取 `software_skills.csv`、`work_activities.csv`、`knowledge.csv`、`essential_skills.csv` 或 `transferable_skills.csv`，因此无法证明“当前值来自 O*NET 31.0 CSV”。 + +**影响** + +当前数据库中的 47/47 技能来源和 137/137 Importance 已由本次复审全量核对为真实；但数据正确性只存在于当前 SQL 快照，没有被自动化测试保护。未来修改若把未抽样的来源替换为格式合法但不存在的三件套，或把未抽样的 Importance 改成错误值，测试仍会通过，随后直接影响词云排序和匹配结果。 + +**修复方向** + +- 增加覆盖全部 47 条 O*NET 技能来源的精确来源清单或 CSV 驱动断言。 +- 增加覆盖全部 137 条非空 Importance 的精确映射断言；若 CSV 不进入正常测试运行环境，至少提交一份受版本控制的完整映射基线，并校验行数、键集合和值。 +- 对 137/137 条映射逐项验证 `SOC + Element ID + Data Value`，而不是只验证非空和应用层来源版本。 + +## 未决问题 + +### Minor-1：每个分类至少关联一个技能未进入断言 + +`SkillCloudSeedDataIntegrationTest.java:89-113` 验证完整分类清单，但只对职位的技能总数和非空热度总数要求 `>= 10`，没有验证每个分类的关联技能数 `>= 1`。当前 68/68 分类均通过独立数据审计，因此不阻断；后续若新增或删改分类，建议补一条分组断言。 + +### Minor-2：Importance 越界值测试仍缺失 + +V35:102-104 定义了 0-5 的 CHECK，但 T3 测试覆盖的是 provenance 缺失、代理标记非法和成功带来源更新，没有直接断言 `importance < 0` 或 `> 5` 在 H2 `MODE=MySQL` 下失败。MySQL 8.4 会执行 CHECK，当前没有发现生产不可兼容语法，但真实 MySQL 尚未在本次环境中执行。 + +### Minor-3：执行计划中的契约哈希未同步修订版 + +当前 `execution-contract.md` 的 SHA-256 为 `98d869bc3b7e29b6fefffd973e561be50ba21ef4f59f4146ccd47cf8164957da`，与 `.spec-superflow.yaml` 一致；但 `.superpowers/sdd/execution-plan.yaml` 仍记录旧哈希 `9ae4be4a8561d7f303804c84d75061bbf2c8b3be2e7c79495689c19ba09d028e`。不影响 T3 数据正确性，建议在 W1 复评通过后同步执行计划。 + +## 未阻断的确认项 + +- V35 的新增列、provenance CHECK、代理标记 CHECK 在 H2 `MODE=MySQL` 下通过测试。 +- `importance_source_code`、`importance_element_id` 的字段长度足以容纳当前 O*NET SOC 和 Element ID。 +- V36 保持纯数据迁移，未提前引入 T4 Java 业务代码。 +- 未修改 V1-V34。 + +## 阻断项汇总 + +T3 当前不通过。第一轮 C-1、I-1、I-2 的阻断已关闭，I-3 的当前数据已清除，但对应的自动化回归保护仍不足，必须补齐 Important-1 后才能交给 T4。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3.md new file mode 100644 index 0000000..6567421 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W1-T3.md @@ -0,0 +1,155 @@ +# W1-T3 独立评审报告 + +## 状态 + +完成。本次评审只读取实现、来源数据、契约和日志，并独立执行测试；未修改任何实现文件，仅写入本报告。 + +## Verdict + +**FAIL** + +存在 1 项 Critical 和 3 项 Important。当前 V36 虽然能在 H2 下通过既有测试，但没有落实已批准的“ESCO 负责职位—技能—分类主关系”方案，且部分金融职位使用与目标职位不符的 O*NET SOC 替代来源；同时 T1 清单中的多个技能分类没有导入。上述问题均阻断 T4。 + +## 评审对象与依据 + +评审对象： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\resources\db\migration\V36__skill_cloud_seed.sql` +- `D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillCloudSeedDataIntegrationTest.java` + +对照依据： + +- `changes/n03-skill-cloud/execution-contract.md` +- `changes/n03-skill-cloud/design.md` +- `changes/n03-skill-cloud/tasks.md` +- `changes/n03-skill-cloud/data/source-manifest.md` +- `docs/skill-taxonomy-sources.md` +- `V35__skill_cloud_taxonomy.sql` +- 本地 ESCO lookup、O*NET 31.0 CSV、实现者 RED/GREEN 日志 + +## 独立检查结果 + +### 1. 数量与基本约束 + +独立解析和 H2 测试结果一致： + +| 项目 | 结果 | +| --- | ---: | +| 行业 | 3 | +| 职位 | 10 | +| 技能分类 | 50 | +| 技能 | 53 | +| 技能别名 | 53 | +| 职位技能关系 | 137 | +| 非空 Importance | 133 | +| 空 Importance | 4 | + +每个职位均有 10 至 17 条职位技能关系，非空 Importance 数量均不少于 10。全部技能有中文标准名、非空 `source_code` 和至少一个别名；`normalized_alias` 无重复。当前 137 条关系的 `category_id` 均与同一 `occupation_id` 对应，未发现现有种子内的跨职位分类错配。 + +### 2. ESCO 与 O*NET 来源核验 + +- V36 的 25 个 `source_type='ESCO'` URI 均可从 `changes/n03-skill-cloud/data/esco-skill-lookup.json` 找到，英文标题与 lookup 一致。 +- V36 的 28 个 O*NET 技能来源三元组 `O*NET-SOC|Workplace Example|Element ID` 均能在 `software_skills.csv` 精确命中；未发现伪造组合。 +- V36 注释声明的 43 组（SOC、Importance、语义元素）映射值均能在本地 O*NET CSV 中命中。其中 `Scikit-learn` 的当前工作区值已按 CSV 精确大小写修复。 +- lookup 中本身存在明显误匹配，例如 Angular→computer programming、Spring Boot→install deicing boot、Node.js→apply electrolytes to cathodes and anodes、React→react to emergency situations in a live performance environment、Redis→redistribute wagered money、Microsoft Power BI→Microsoft Access。当前 V36 没有使用这些错误 URI，因此它们不是本批次已导入数据的直接错误，但该 lookup 不能作为未经复核的可信自动映射源。 + +### 3. 独立测试 + +执行命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest' test +``` + +结果： + +```text +SkillCloudSeedDataIntegrationTest: Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyMigrationTest: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 +Total: Tests run: 7, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +该结果与实现者 GREEN 日志一致。 + +## Findings + +### Critical + +#### C-1：V36 用 O*NET/项目映射替代了已批准的 ESCO 职位—技能—分类主关系 + +证据： + +- 执行契约第 55 行要求“使用 ESCO 作为职位—技能—分类主关系，O*NET Importance 作为 heat”。 +- 设计 D1 第 32-42 行同样规定 ESCO 负责职位—技能—分类关系；来源清单第 34-39、51、60-63 行要求保留原上游来源 ID，并由项目维护跨源重要性映射。 +- V35 第 34 行将 `skill_occupations.source_code` 定义为 `ESCO 职位来源 ID`。 +- V36 第 30-39 行将 10 个职位的 `source_code` 全部写成 O*NET SOC，并将 `source_version_id` 指向 O*NET 来源版本。 +- V36 第 203-340 行 137 条职位技能关系的 `source_version_id` 全部为 O*NET。 +- 本地只有 ESCO 技能 lookup，没有 ESCO 职位—技能关系数据；V36 已导入的 25 个 ESCO 技能只提供了技能 URI，未提供任何 ESCO 职业关系。 + +影响：这不是字段命名的小问题，而是数据治理来源发生实质变化：职位节点被标成 O*NET SOC，职位技能关系改由 O*NET software_skills、项目分类和代理 Importance 人工组装。下游接口即使运行正确，也无法证明返回的是契约批准的 ESCO 主关系。该变化命中执行契约第 136 行的 Rewind 条件“数据来源发生实质变化”。 + +### Important + +#### I-1：两个金融职位使用不能代表目标职位的替代 SOC，且替代映射未获批准或写入来源清单 + +证据： + +- V36 第 34 行将“金融数据分析师”写成 `13-2031.00`。本地 `occupation_data.csv` 中该 SOC 的标题是 **Budget Analysts**；原计划目标 `13-2051.00` 是 **Financial and Investment Analysts**。 +- V36 第 35 行将“风控建模工程师”写成 `13-2053.00`。本地 CSV 中该 SOC 的标题是 **Insurance Underwriters**；原计划目标 `13-2054.00` 是 **Financial Risk Specialists**。 +- V36 第 16-17 行仅以 SQL 注释说明替换；`source-manifest.md` 第 22-23 行没有批准代理 SOC，也没有记录替换后的目标语义。 +- `13-2051.00` 和 `13-2054.00` 在本地四份 Importance CSV 中确实没有行，但“原目标无数据”不等于可以用语义不同的职业覆盖其来源 ID。 + +影响：Budget Analysts 与 Insurance Underwriters 的 O*NET 活动分布不能代表金融数据分析师和风控建模工程师。尤其是 Insurance Underwriters 作为风控建模工程师的热度来源，会导致技能热度与目标职位脱节。必须在来源清单和契约中明确代理规则与语义合理性，或重新选择可代表目标职位的来源；不能直接把替代 SOC 写入目标职位的 `source_code`。 + +#### I-2：T3 清单中的多个技能分类未落表，测试只要求至少一个分类因而漏检 + +证据： + +- `source-manifest.md` 第 18-27 行给出了每个职位的首批技能分类范围，第 30 行明确这些分类是“每个职位首批应覆盖的能力域”。 +- V36 第 42-91 行为每个职位固定导入 5 个分类，共 50 个。 +- 前端开发缺少清单中的“浏览器与调试”“测试”。 +- DevOps 缺少“配置管理与 IaC”“可观测性”“安全”等独立能力域。 +- 风控建模缺少“模型验证”等能力域。 +- 教学设计师缺少“无障碍与规范”等能力域。 +- 测试第 70-71 行只断言 `category_count >= 1`，因此即使只导入一个分类也会通过。 + +影响：已批准的 T3 分类覆盖范围没有完整交付，后续接口会缺少部分可筛选分类；当前测试无法阻止该类回归。 + +#### I-3：Importance 缺少逐技能来源映射和有效验证，现有数据大量依赖同值代理 + +证据： + +- V36 第 5-15 行说明所有技术技能使用“Working with Computers”等职业活动代理值，而不是 O*NET software_skills 自身的 Importance。 +- V36 第 203-340 行的 133 条非空关系实际只形成 39 个不同的 `(occupation, importance)` 组合；多个技能共享同一代理值，例如同一职业内多数技术技能全部为同一个值。 +- V35 第 93-95 行只保存 `importance` 和 `source_version_id`，没有保存用于推导热度的 O*NET Element ID；V36 的注释不是机器可追溯的数据。 +- 测试第 106-127 行只精确校验前端 JavaScript 的 `4.88` 和 Figma 的 `null`，其余 132 条重要性记录或错误来源元素不会被发现。 + +影响：词云热度会因大量同值技能而主要退化为 `nameZh ASC` 排序，无法证明每个技能的热度确实来自对应 O*NET 来源。若继续该口径，应新增可追溯的 Importance 映射记录并扩大测试，或者重新定义可接受的代理规则和排序语义。 + +### Minor + +1. **“45 个 Importance 原值”的核验数量不实。** V36 第 5-15 行声明了 43 个职业/能力域映射，133 条非空关系只有 39 个不同 `(occupation, importance)` 组合；报告中“45 个 Importance 原值”没有可复现证据。这是核验报告计数问题，不代表已发现任意 CSV 伪造值。 + +2. **最终提交可能漏掉 Scikit-learn 大小写修复。** 当前工作区是 `Scikit-learn`，与 CSV 一致；但 `git status` 显示 V36 为 `AM`，即该修复尚未 staged。提交前必须确保修正包含在最终 T3 提交中。 + +3. **测试主要验证行数、非空值和两个单点值，不是完整来源契约测试。** 当前即使将除 JavaScript 外的 Importance 改错、将非空 ESCO/O*NET source code 改成任意非空字符串，或只保留每个职位一个分类，测试仍可能通过。分类跨职位错配也不会被测试发现，因为数据库没有覆盖 `(occupation_id, category_id)` 的复合外键。 + +## 已确认没有问题的部分 + +- V36 是纯数据迁移：只有 7 个 `INSERT`，未包含 `UPDATE/DELETE/DROP/ALTER/CREATE`。 +- `git diff HEAD` 在本任务实现范围内只显示 V36 和 T3 测试；V1-V35 未修改，`src/main/java/com/mydreamjob/skill` 不存在，未提前实现 T4 业务代码。 +- 当前 137 条关系的技能分类均与所属 occupation 对应；所有技能、别名、来源版本和关系均满足 V35 现有外键及唯一约束。 +- 28 个 O*NET 技能来源组合和 25 个 ESCO URI 均可从本地来源文件核实，没有发现伪造 O*NET software row 或伪造 ESCO URI。 + +## 未决问题 + +- 必须决定是恢复 ESCO 职位—技能—分类主关系，还是正式修改 design、source manifest 和执行契约，承认以 O*NET SOC 与项目映射作为主关系。后者属于数据来源实质变化，应走 Rewind。 +- 必须确认“代理 Importance”是否属于已批准方案；若属于，需记录每个技能或能力域的 O*NET Element ID、代理理由和质量规则，并重新定义同值排序的产品语义。 +- 若继续使用 `13-2031.00` 和 `13-2053.00`，需要提供可审查的职位语义映射依据并更新来源清单；否则应更换代表 SOC。 +- 本机未进行真实 MySQL 迁移验证，仅验证 H2 `MODE=MySQL`。该限制沿用 T2 的未决项。 + +## 阻断项 + +T3 不通过。C-1、I-1、I-2、I-3 均必须在开始 T4 前处理；其中 C-1 和 I-1 优先触发规划/契约 Rewind 或获得明确的来源变更批准。既有 7 项测试通过不能覆盖这些来源和分类语义问题。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T4-rereview.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T4-rereview.md new file mode 100644 index 0000000..02e18ff --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T4-rereview.md @@ -0,0 +1,130 @@ +# W2-T4 修复复审报告 + +## 状态 + +完成。本轮仅复审修复提交 `52751e5`，对照第一轮报告 `W2-T4.md`，未修改任何实现、测试、迁移或配置文件；仅新增本报告。 + +## Verdict + +**PASS** + +第一轮阻断项 `W2-T4-I1` 已关闭。两个投影查询均已保证技能分类的 `occupation_id` 与职位技能关系的 `occupation_id` 一致，新增测试真实构造了跨职位分类错配数据并验证查询排除该数据。规定回归集全部通过，未发现新的 Critical、Important 或阻断项。 + +## Finding 关闭状态 + +### W2-T4-I1：分类过滤查询未强制 category 与 occupation 属于同一职位 + +状态：**已关闭**。 + +关闭证据： + +- 整职位查询在分类关联后增加了 `c.occupationId = os.occupationId`： + `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:36-40` +- 职位加分类查询同样增加了该约束，并保留 `os.occupationId = :occupationId` 与 `os.categoryId = :categoryId`： + `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:63-69` +- 两个查询均继续通过 `os.skillId = s.id` 和 `os.categoryId = c.id` 连接技能及分类，未改变投影字段或原过滤条件。 + +## 独立测试结果 + +环境： + +- `JAVA_HOME=D:\java\jdk` +- Java：OpenJDK 17.0.16 +- Maven：`D:\maven\apache-maven-3.9.7\bin\mvn.cmd` +- 工作目录：`D:\1\zhi\zhilai\mydreamjob-server` + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest' test +``` + +结果： + +```text +SkillCloudSeedDataIntegrationTest: Tests run: 8, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyMigrationTest: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyRepositoryTest: Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 +Total: Tests run: 16, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 40.769 s +``` + +## 必须检查项逐项结论 + +### 1. 两个投影查询均约束 category 所属 occupation，且保留过滤：通过 + +两个查询都在连接 `os.categoryId = c.id` 后增加： + +```sql +AND c.occupationId = os.occupationId +``` + +整职位查询继续只按目标 `occupationId` 过滤；职位加分类查询继续同时按目标 `occupationId` 和 `categoryId` 过滤。因此： + +- 不会再通过职位 A 的关系读取仅属于职位 B 的分类； +- 分类参数仍必须与关系中的分类 ID 完全相等； +- 职位参数语义未改变。 + +位置： + +- `OccupationSkillRepository.java:36-43` +- `OccupationSkillRepository.java:63-72` + +### 2. 新增测试真实插入跨职位错配关系，且可证伪修复：通过 + +新增测试位于 `D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillTaxonomyRepositoryTest.java:123-152`。 + +测试通过 `JdbcTemplate` 向真实 `occupation_skills` 表插入： + +- `occupation_id = 1` +- `category_id = 201` +- `skill_id = 1` +- `source_version_id = 2` + +其中分类 `201` 在 V36 中明确属于职位 `2`，见 `V36__skill_cloud_seed.sql:67`；技能 `1` 存在，见 `V36__skill_cloud_seed.sql:129`。插入使用合法外键值和完整来源字段，因此构造的是数据库允许保存、但职位与分类不属于同一节点的逻辑错配关系。 + +该测试没有断言 SQL 文本，而是调用真实 Repository 方法并检查返回结果。修复前的查询缺少同职位约束，插入关系会满足 `os.skillId = s.id`、`os.categoryId = c.id` 和 `os.occupationId = 1`，因此： + +- 整职位查询会包含技能 `1`，在第 `142` 行 `doesNotContain(1L)` 失败； +- `occupation=1, category=201` 查询会包含该关系，在第 `144-146` 行失败。 + +修复后该关系被 `c.occupationId = os.occupationId` 排除，测试通过，可证伪性成立。 + +测试方法带 `@Transactional`，见 `SkillTaxonomyRepositoryTest.java:123-124`，插入数据在测试结束后回滚，不污染其他测试或 V36 计数。 + +### 3. 三项断言均正确覆盖：通过 + +`SkillTaxonomyRepositoryTest.java:136-151` 分别验证： + +- 整职位查询仍包含正常技能 `3`，且不包含错配技能 `1`；当前断言在 `139-142` 行。 +- 错误分类组合 `occupation=1, category=201` 返回空列表；当前断言在 `144-146` 行。 +- 正确组合 `occupation=2, category=201` 仍返回技能 `1`，证明修复没有误伤合法数据；当前断言在 `148-151` 行。 + +三项断言覆盖了第一轮要求的安全边界和正常路径。 + +### 4. 正常查询、空 Importance、来源字段和计数：通过 + +- 原整职位和分类过滤测试继续验证职位 `1` 返回 15 项、分类 `101` 返回精确技能集合，以及错误职位分类返回空：`SkillTaxonomyRepositoryTest.java:95-120`。 +- O*NET 来源字段、`importance`、`importanceSourceCode`、`importanceElementId`、代理标记和排序字段的原断言仍通过：`SkillTaxonomyRepositoryTest.java:154-188`。 +- Figma 的空 Importance 场景仍通过：`SkillTaxonomyRepositoryTest.java:176-188`。 +- V36 计数仍为来源版本 2、行业 3、职位 10、分类 68、技能 72、别名 72、职位技能关系 139：`SkillTaxonomyRepositoryTest.java:72-81`。 +- 指定回归集 16 项全部通过，未观察到由新增约束造成的正常查询、投影字段或种子数据回归。 + +### 5. 提交范围与 T5 边界：通过 + +`git diff --name-only 0c77824..52751e5` 仅列出： + +- `mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/OccupationSkillRepository.java` +- `mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillTaxonomyRepositoryTest.java` + +未修改 V35/V36、生产配置、安全配置或其他模块，也未实现任何 T5 排名、匹配或学习路径代码。`git diff --check 0c77824..52751e5` 通过。 + +## 剩余阻断项 + +无。 + +## 非阻断备注 + +V35 仍允许数据库层面写入 `occupation_id` 与 `category_id` 逻辑不一致的关系，因为当前外键是单列外键；本次修复保证 Repository 查询不会返回此类错配数据。是否需要增加数据库复合完整性约束属于独立的数据模型变更，不影响 T4 通过。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T4.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T4.md new file mode 100644 index 0000000..e2ace83 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T4.md @@ -0,0 +1,150 @@ +# W2-T4 独立评审报告 + +## 状态 + +完成。本轮只评审提交 `0c77824`、修订后的契约与规划产物、V35/V36，并独立执行指定测试；未修改实现、测试、迁移或配置文件，仅写入本报告。 + +## Verdict + +**FAIL** + +未发现 Critical。发现 1 项 Important：技能投影查询没有保证 `occupation_skills.category_id` 对应的分类属于同一个 `occupation_id`，在数据库出现逻辑错配关系时会返回跨职位数据。该问题会阻断 T4 通过，修复并复审后可进入 T5。 + +## 独立测试输出 + +环境： + +- `JAVA_HOME=D:\java\jdk` +- Java：OpenJDK 17.0.16 +- Maven：`D:\maven\apache-maven-3.9.7\bin\mvn.cmd` +- 工作目录：`D:\1\zhi\zhilai\mydreamjob-server` + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest' test +``` + +结果： + +```text +SkillCloudSeedDataIntegrationTest: Tests run: 8, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyMigrationTest: Tests run: 2, Failures: 0, Errors: 0, Skipped: 0 +SkillTaxonomyRepositoryTest: Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 +Total: Tests run: 15, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 44.358 s +``` + +## 必须检查项逐项结论 + +### 1. 实体与 V35 映射：通过 + +7 个实体均映射到 V35 对应表，主键为 `Long` + `GenerationType.IDENTITY`。已逐表核对表名、列名、长度、可空性和类型： + +- `SkillSourceVersion` 与 `skill_source_versions`：字段、长度和可空性一致。 +- `SkillIndustry` 与 `skill_industries`：字段、长度和可空性一致。 +- `SkillOccupation` 与 `skill_occupations`：字段、长度和可空性一致。 +- `SkillCategory` 与 `skill_categories`：字段、长度和可空性一致。 +- `Skill` 与 `skills`：字段、长度和可空性一致。 +- `SkillAlias` 与 `skill_aliases`：字段、长度和可空性一致；该表无 `updated_at`，实体也未映射该列。 +- `OccupationSkill` 与 `occupation_skills`：`DECIMAL(4,2)`、来源字段长度、`TINYINT(1)` 代理标记及可空性匹配；`importanceIsProxy` 使用 `@JdbcTypeCode(SqlTypes.TINYINT)`。 + +本次测试在 `ddl-auto=validate` 下执行通过，证明 V35 七张表实际参与了 Hibernate 校验。 + +### 2. Schema Filter 与生产配置：通过 + +`SkillTaxonomyRepositoryTest` 的 validate filter 明确包含全部七个 V35 表： + +`D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillTaxonomyRepositoryTest.java:182-231` + +因此该 filter 隔离的是仓库历史中与本任务无关的表，不会跳过 V35 实体的列类型或可空性校验。测试通过 `@TestPropertySource` 覆盖测试数据库和 `ddl-auto`，提交未修改生产 `application.yml`、`SecurityConfig` 或其他配置。 + +### 3. 行业、职位、分类关联校验：通过 + +以下 Repository 方法正确表达了单节点关系约束： + +- `SkillOccupationRepository.existsByIdAndIndustryId` / `findByIdAndIndustryId` +- `SkillCategoryRepository.existsByIdAndOccupationId` / `findByIdAndOccupationId` + +测试覆盖了正确组合、跨行业职位、跨职位分类和不存在的 ID： + +`D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillTaxonomyRepositoryTest.java:78-88` + +### 4. occupation + 可选 category 查询的同职位约束：不通过 + +两个投影查询都只连接 `os.category_id = c.id`，没有断言 `c.occupation_id` 与 `os.occupation_id` 相同： + +`D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:36-40` + +`D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:62-66` + +V35 对 `occupation_skills.category_id` 只有单列外键，不阻止一条关系的 `occupation_id` 指向职位 A、`category_id` 指向职位 B 的分类： + +`D:\1\zhi\zhilai\mydreamjob-server\src\main\resources\db\migration\V35__skill_cloud_taxonomy.sql:114-117` + +这违反了契约要求“分类必须属于职位”： + +`D:\1\zhi\zhilai\changes\n03-skill-cloud\execution-contract.md:59` + +也违反了设计中的“category_id 必须属于 occupation_id”： + +`D:\1\zhi\zhilai\changes\n03-skill-cloud\design.md:119` + +现有测试 `findSkillProjectionsByOccupationIdAndCategoryId(2L, 101L)` 只证明当前种子数据中不存在 `职位 2 + 分类 101` 关系，不能发现上述缺列条件；即使插入一条 `occupation_id=2, category_id=101` 的逻辑错配关系，该查询仍会返回它： + +`D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillTaxonomyRepositoryTest.java:113-115` + +修复方向明确：两个查询均增加分类所属职位约束，例如 `AND c.occupationId = os.occupationId`；同时增加可证伪测试，用跨职位分类关系验证查询不会返回数据。 + +### 5. 投影 14 个字段：通过 + +`SkillCloudSkillProjection` 包含 14 个字段，覆盖技能身份、名称、来源、分类、`importance`、`importanceSourceCode`、`importanceElementId`、`importanceIsProxy`、技能排序值和分类排序值。构造表达式与字段顺序一致，`importance` 使用 `BigDecimal`，代理标记使用 `Boolean`。 + +`D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\dto\SkillCloudSkillProjection.java:5-20` + +### 6. 未提前实现 T5–T8：通过 + +提交只新增 7 个实体、1 个投影、7 个 Repository 和专项测试；未新增排名服务、匹配服务、学习路径服务、Controller、安全规则或 OpenAPI 变更。Repository 查询明确未实现最终排序，符合 T5 边界。 + +### 7. 测试数据覆盖与可证伪性：除跨职位查询外通过 + +测试使用了 V36 种子数据，覆盖计数、关联校验、整职位查询、可选分类过滤、O*NET 来源、空 Importance、代理标记、别名和按 ID 查询。查找均为精确断言，不是仅计数。 + +但第 4 项的跨职位分类错配缺少可证伪用例；即使 Repository 查询遗漏同职位条件，当前 15 项测试仍全部通过。 + +### 8. 提交范围与历史迁移：通过 + +提交只包含允许的 16 个文件。独立哈希核对确认 V35/V36 在 `0c77824` 中未发生变化，提交未修改其他模块、已有迁移、生产配置或安全配置。 + +## Findings + +### Critical + +无。 + +### Important + +#### W2-T4-I1：分类过滤查询未强制 category 与 occupation 属于同一职位 + +证据： + +- `OccupationSkillRepository` 两个查询均只有 `os.categoryId = c.id`，缺少 `c.occupationId = os.occupationId`： + - `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:36-40` + - `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:62-66` +- V35 外键不提供复合 `(occupation_id, category_id)` 完整性： + - `D:\1\zhi\zhilai\mydreamjob-server\src\main\resources\db\migration\V35__skill_cloud_taxonomy.sql:114-117` +- 契约明确要求分类必须属于职位： + - `D:\1\zhi\zhilai\changes\n03-skill-cloud\execution-contract.md:59` + - `D:\1\zhi\zhilai\changes\n03-skill-cloud\design.md:119` + +影响：一旦数据写入出现逻辑错配，整职位查询会返回其他职位的分类技能；带分类查询也可能在对应关系存在时返回跨职位数据，破坏词云筛选边界，并影响后续匹配和学习路径。当前种子数据恰好一致，因此测试未暴露问题。 + +### Minor + +无额外阻断项。 + +## 未决问题 + +无。修复语义明确：查询必须同时绑定 category 所属 occupation；修复后应增加一条跨职位分类关系的失败用例并重新评审。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T5-rereview.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T5-rereview.md new file mode 100644 index 0000000..82f570c --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T5-rereview.md @@ -0,0 +1,94 @@ +# W2-T5 聚焦修复独立复审报告 + +- **评审对象**：T5 I-1 聚焦修复工作区差异（基于 `e3eecfe`） +- **前次评审**：`changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T5.md`，结论 `FAIL` +- **复审范围**：仅检查 `SkillCloudItem.java`、`SkillCloudQueryService.java`、`SkillCloudQueryServiceTest.java` 及其对既有行为的影响 +- **结论**：**PASS** +- **剩余阻断项**：无 + +## Finding 关闭状态 + +### [I-1] 响应项遗漏技能来源标识：已关闭 + +修复已同时补齐 DTO 字段和 Service 映射： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\dto\SkillCloudItem.java:11-12` + - 新增 `sourceType` + - 新增 `sourceCode` +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\SkillCloudQueryService.java:102-117` + - `toItem(...)` 在 `nameEn` 后依次映射 `projection.sourceType()` 和 `projection.sourceCode()`（第 109-110 行） + - 热度来源仍独立映射为 `heatSourceCode`、`heatElementId`、`heatProxy`（第 114-116 行） + +字段语义未混淆：技能自身来源是 `sourceType/sourceCode`，O*NET Importance 来源是 `heatSourceCode/heatElementId/heatProxy`。 + +### 双来源测试与可证伪性：已关闭 + +`D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillCloudQueryServiceTest.java:198-236` 同时锁定两类来源： + +- O*NET 技术来源：第 219-220 行精确断言 + - `sourceType = "O*NET"` + - `sourceCode = "15-1254.00|React|2.E.7.h"` +- ESCO 技能来源：第 232-234 行精确断言 + - `sourceType = "ESCO"` + - `sourceCode = "http://data.europa.eu/esco/skill/3cd569a2-4f88-4c1e-9995-8dce8c5e51a7"` +- 来源隔离：第 235 行额外断言 JavaScript 的 `heatSourceCode = "15-1254.00"`，与 ESCO `sourceCode` 明显不同。 + +因此： + +- 若 `sourceType` 或 `sourceCode` 映射被遗漏并传为 `null`，上述精确断言会失败。 +- 若二者被交换，O*NET 与 ESCO 的精确值断言会失败。 +- 若技能来源与热度来源混用，React 的完整 O*NET 三元组断言及 JavaScript 的 URI/SOC 对照断言会失败。 + +## 原有行为回归 + +独立执行指定环境与命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest' test +``` + +结果： + +```text +Tests run: 24, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +Surefire 分类计数： + +- `SkillTaxonomyMigrationTest`：2 +- `SkillCloudSeedDataIntegrationTest`：8 +- `SkillTaxonomyRepositoryTest`：6 +- `SkillCloudQueryServiceTest`：8 + +已核验原有行为： + +- 全量查询、连续排名：`SkillCloudQueryServiceTest.java:62-81` +- 热度降序、同热度名称排序、null heat 末位：`SkillCloudQueryServiceTest.java:84-107` +- 同值排序的 `skillId` 稳定性：`SkillCloudQueryServiceTest.java:109-145` +- 分类过滤及缺省分类：`SkillCloudQueryServiceTest.java:166-179` +- 合法空分类返回空列表：`SkillCloudQueryServiceTest.java:181-196` +- 必填/正整数/未知节点/跨行业职位/跨职位分类校验：`SkillCloudQueryServiceTest.java:238-264` +- provenance 字段映射：`SkillCloudQueryServiceTest.java:198-236` + +本次修复没有改变排序、过滤、校验或热度来源映射逻辑，仅扩展响应字段与测试断言。 + +## 变更范围 + +自 `e3eecfe` 起，T5 实现差异仅包含： + +- `SkillCloudQueryService.java` +- `SkillCloudItem.java` +- `SkillCloudQueryServiceTest.java` + +共 `20 insertions, 3 deletions`。未修改 V35/V36、Repository、安全配置或应用配置；未发现 T6/T7/T8 的 Controller、匹配度 Service、学习路径 Service 或权限配置实现。 + +## 非阻断备注 + +- **M-1 仍为 Minor**：设计文档中的 `heatSource` 嵌套形态与当前三个扁平热度来源字段尚未统一，按前次评审要求留待 T8 明确 OpenAPI 和接口文档。 +- **M-2 仍为 Minor**：Service 层未单独断言无 heat 技能的 `sourceType/sourceCode`；Repository 层已在该场景断言 Figma 保留来源（`SkillTaxonomyRepositoryTest.java:183-188`），且 Service 映射无条件执行，不构成阻断。 + +## 最终裁决 + +`PASS`。I-1 已关闭，指定联合回归全部通过，未发现新的 Critical 或 Important 问题；T5 可进入后续任务门禁。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T5.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T5.md new file mode 100644 index 0000000..c33d614 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T5.md @@ -0,0 +1,94 @@ +# W2-T5 独立评审报告 + +- **评审对象**：`e3eecfe feat: add skill cloud query service` +- **评审基线**：`52751e5`（T4 复审通过提交） +- **结论**：**FAIL** +- **评审范围**：仅检查提交 `e3eecfe` 的 5 个文件；未修改实现文件。 + +## 独立测试结果 + +环境：`JAVA_HOME=D:\java\jdk`，Maven `D:\maven\apache-maven-3.9.7\bin\mvn.cmd`。 + +执行： + +```powershell +mvn -Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest test +``` + +结果： + +```text +Tests run: 24, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +测试数量拆分为：迁移 2、种子数据 8、Repository 6、词云 Service 8。 + +## Important + +### [I-1] 响应项遗漏技能自身的来源标识，违反 T5 的“来源字段”和可追溯要求 + +`SkillCloudSkillProjection` 已查询并携带技能来源字段： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\dto\SkillCloudSkillProjection.java:10-11` + - `sourceType` + - `sourceCode` +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:25-26` + - 查询投影包含 `s.sourceType` 和 `s.sourceCode` +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\repository\OccupationSkillRepository.java:52-53` + - 分类查询同样包含这两个来源字段 + +但 Service 转换成响应 DTO 时丢弃了这两个字段： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\SkillCloudQueryService.java:102-115` + - `toItem(...)` 没有传入 `projection.sourceType()` 或 `projection.sourceCode()` +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\dto\SkillCloudItem.java:5-17` + - Item 只有 `heatSourceCode`、`heatElementId`、`heatProxy`，没有技能来源类型或技能来源编码 + +这会使返回结果无法在接口层区分某个技能来自 ESCO 还是 O*NET `software_skills`，只剩通过数据库内部 ID 才能反查来源。T5 明确要求返回“来源字段”： + +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\tasks.md:21` +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\execution-contract.md:41` +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\specs\skill-cloud\spec.md:86-93` + +当前测试也没有覆盖该缺口：`SkillCloudQueryServiceTest` 只断言了 React 的 `heatSourceCode`、`heatElementId` 和 `heatProxy`（`SkillCloudQueryServiceTest.java:198-223`），没有断言 Item 中应携带 `sourceType`/`sourceCode`。因此该遗漏是可复现但不可被现有测试发现的。 + +**修复要求**：在 `SkillCloudItem` 中补齐技能来源字段，在 `toItem` 中映射，并增加至少一个 ESCO 技能和一个 O*NET `software_skills` 技能的精确断言；同时确认无 heat 的技能仍保留其技能来源。 + +## Minor + +### [M-1] `heatSource` 的响应形态与设计文档不一致，需在 T8 前统一 + +设计文档将词云项描述为包含 `heatSource`： + +- `D:\1\zhi\zhilai\changes\n03-skill-cloud\design.md:131` + +当前实现使用三个扁平字段表达同一语义： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\dto\SkillCloudItem.java:13-16` + - `heatSourceCode` + - `heatElementId` + - `heatProxy` + +这不是当前 T5 的运行时错误，但 T8 必须明确 OpenAPI 和接口文档采用扁平字段还是嵌套 `heatSource` 对象，避免契约漂移。 + +### [M-2] T5 服务测试对 null heat 的来源字段覆盖不足 + +`SkillCloudQueryServiceTest` 对 React 的非空 heat 来源做了精确断言，但没有对无 heat 技能（如当前种子数据中的 Figma）断言 `heatSourceCode`、`heatElementId` 和 `heatProxy` 的最终 DTO 值。T4 投影测试覆盖了该场景，因此暂不阻断；修复 I-1 时应顺带补齐服务层断言。 + +## 已通过检查 + +- 行业存在性、职位存在性、行业—职位归属校验均使用 `BusinessException(ErrorCode.BAD_REQUEST)`。 +- 非法 `industryId`、`occupationId`、`categoryId`（null、0、负数）均返回 400 语义；未知节点和跨行业/跨职位节点同样返回 400。 +- `categoryId` 缺省时调用整职位查询；传入时调用职位+分类查询，并保留 Repository 的职位—分类归属约束。 +- 排序实现为 `heat DESC`、null heat 最后、`nameZh ASC`、`skillId ASC`；当前种子数据测试能捕获名称比较器和 skillId 比较器写错，也覆盖了 null heat 末位和连续 rank。 +- 空分类返回成功、`total=0`、空 items。 +- `SkillCloudResponse` 对列表执行 `List.copyOf`，响应列表不可变。 +- 提交范围仅包含 T5 的 Service、3 个 DTO 和测试文件，没有实现 T6/T7/T8，也没有修改 V35/V36、安全配置或既有接口。 +- 未发现事务、空指针、可变集合、随机性或明显性能问题；当前静态词库规模下三次校验查询加一次投影查询是合理的。 + +## 未决问题 + +1. 技能来源字段的最终 JSON 名称应为 `sourceType`/`sourceCode`，还是采用带 `skillSource` 前缀的名称；需在修复 I-1 和 T8 OpenAPI 时统一。 +2. `heatSource` 是否需要改成嵌套对象；若保持扁平字段，应同步修订设计文档和后续接口文档。 +3. I-1 修复后必须重新执行本报告使用的 24 项联合回归，并重新评审；T5 评审通过前不得进入 T6。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T6-rereview.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T6-rereview.md new file mode 100644 index 0000000..85374e8 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T6-rereview.md @@ -0,0 +1,98 @@ +# W2-T6 测试隔离修复独立复审报告 + +## Verdict + +**PASS** + +T6 原 Review 中唯一 Important finding 已关闭。新增的三个隔离测试均通过服务返回结果进行断言，能够分别证伪 `nameZh` 分支、alias 匹配链路和 alias 归属隔离的相关回归；类级事务确保测试数据回滚。未发现新的阻断项，T6 可进入下一任务。 + +## Finding 关闭状态 + +### T6-I1：`nameZh` 与 alias 匹配遗漏无法证伪 + +**状态：已关闭。** + +1. **nameZh-only 隔离成立** + - 测试：`matchesOnlyNameZhForV36JavaWhenOtherIdentityFieldsDoNotMatch` + - 位置：`D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillMatchServiceTest.java:237-252` + - 第 239 行先将 V36 Java 的 `code` 改为 `JAVA_NAME_ZH_ISOLATED`。 + - Java 的 V36 `nameZh` 为 `Java`，`nameEn` 为 `Java (computer programming)`，alias 为 `Java (computer programming)`；学生输入仅为 `Java`。 + - 因此学生输入只能匹配 `nameZh`。删除生产代码的 `nameZh` 分支后，Java 必然进入 `missingSkills`，方法第 246-251 行会失败。 + +2. **alias-only 隔离成立** + - 测试：`matchesOnlyInsertedAliasForV36JavaWhenAliasIsTheOnlyIdentityMatch` + - 位置：`SkillMatchServiceTest.java:254-272` + - 第 256-259 行插入 Java 专属 alias `ONLY.ALIAS-JVM-XYZ`，与目标技能的 `nameZh`、`nameEn`、`skillCode` 均不相同。 + - 学生输入只有该 alias，并直接断言服务返回的 `matchedSkills` 包含 Java、`missingSkills` 包含 Python。 + - 删除 alias 查询或 alias 匹配分支后，Java 不会命中，测试必然失败。 + +3. **alias 归属隔离成立** + - 测试：`doesNotApplyAliasOwnedByAnotherTopSkillToV36Java` + - 位置：`SkillMatchServiceTest.java:274-292` + - 第 276-279 行将唯一 alias 关联到技能 2（Python），学生只输入该 alias。 + - 断言要求技能 2 命中、技能 1 缺失；若 alias 被错误共享给其他 TOP 技能，技能 1 也会命中，分数的断言会失败。 + +### 事务与回滚 + +**状态：满足要求。** + +- `SkillMatchServiceTest` 在类级使用 `@Transactional`：`SkillMatchServiceTest.java:51-52`。 +- 未发现 `@Commit` 或 `@Rollback(false)`。 +- 三个新增测试的 JDBC 插入/更新和 `service.match(...)` 调用均在测试事务内执行，测试结束后由 Spring Test 默认回滚，不会污染共享 H2 或 V36 数据。 + +### 服务结果断言 + +**状态：满足要求。** + +- 三个新增测试均直接对 `SkillMatchResponse` 的 `matchScore()`、`matchedSkills()` 和 `missingSkills()` 做断言，不是仅验证 helper。 +- alias-only 测试还会在第 265-271 行同时校验命中与缺失列表，能够发现“匹配分数正确但技能集合错误”的实现偏差。 + +## 独立测试结果 + +工作目录：`D:\1\zhi\zhilai\mydreamjob-server` + +环境： + +- `JAVA_HOME=D:\java\jdk` +- Maven：`D:\maven\apache-maven-3.9.7\bin\mvn.cmd` + +执行命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest' test +``` + +结果： + +```text +Tests run: 40, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 47.828 s +``` + +测试组成： + +| 测试类 | 数量 | Failures | Errors | +|---|---:|---:|---:| +| `SkillTaxonomyMigrationTest` | 2 | 0 | 0 | +| `SkillCloudSeedDataIntegrationTest` | 8 | 0 | 0 | +| `SkillTaxonomyRepositoryTest` | 6 | 0 | 0 | +| `SkillCloudQueryServiceTest` | 8 | 0 | 0 | +| `SkillMatchServiceTest` | 16 | 0 | 0 | + +## 回归与范围核验 + +- T6 原有 13 项行为测试全部保留，新增 3 项隔离测试，专项测试总数为 16。 +- `nameEn`、`skillCode`、去重、TOP10、分母、非法 JSON、空数据、排序和跨职位隔离等既有测试均通过。 +- `git diff --name-only -- mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillMatchService.java mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillNameNormalizer.java` 无输出，生产实现相对当前审阅基线未被本次测试修复修改。 +- 源码搜索未发现 `LearningPath`、`SkillCloudController`、`SkillMatchController` 或相关接口实现；T7/T8 未实现。 +- `git diff --check` 未报告代码差异错误；输出中的无关 `.trae` 文档警告不属于 T6 范围。 + +## 剩余阻断项 + +无。 + +## 结论 + +T6-I1 已关闭，T6 测试隔离修复满足可证伪性、事务隔离和服务结果断言要求。允许 T6 进入后续流程，但在流程门禁正式记录 `PASS` 前不得启动 T7。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T6.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T6.md new file mode 100644 index 0000000..e9d4c65 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T6.md @@ -0,0 +1,71 @@ +# W2-T6 独立评审报告 + +## Verdict + +**FAIL** + +T6 生产实现的主流程与当前契约基本一致，但测试未满足本次评审明确要求的可证伪性，尤其无法发现 `nameZh` 匹配或别名匹配被删除的回归。必须先补齐隔离型测试并通过复审，才能进入 T7。 + +## 独立测试结果 + +环境： + +- `JAVA_HOME=D:\java\jdk` +- Maven：`D:\maven\apache-maven-3.9.7\bin\mvn.cmd` +- 工作目录：`D:\1\zhi\zhilai\mydreamjob-server` + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest' test +``` + +结果： + +```text +Tests run: 37, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 58.642 s +``` + +测试组成：迁移 2、种子数据 8、Repository 6、词云查询 8、个人匹配 13。 + +## 阻断项 + +### Important：无法证伪 `nameZh` 与 alias 匹配遗漏 + +文件： + +- `D:\1\zhi\zhilai\mydreamjob-server\src\test\java\com\mydreamjob\skill\SkillMatchServiceTest.java:212-235` +- `D:\1\zhi\zhilai\mydreamjob-server\src\main\java\com\mydreamjob\skill\SkillMatchService.java:137-142` + +问题： + +- 测试方法 `matchesNameZhNameEnSkillCodeAndAliasAndDeduplicatesStudentSkills` 试图一次覆盖 `nameZh`、`nameEn`、`skillCode` 和别名，但没有隔离各候选来源。 +- 技能 1 的 `nameZh` 为“数据库”，同时注册别名 `JVM Language`。学生技能数组同时包含“数据库”和 `jvm-language`。 +- 如果删除 `SkillMatchService.matches(...)` 中的 `nameZh` 判断，技能 1 仍可通过别名命中，测试继续通过。 +- 如果删除 alias 查询、alias 分支或整个别名匹配链路，技能 1 仍可通过 `nameZh` 命中，测试同样继续通过。 +- 因此现有测试不能证明 REQ-SM-3 和 TO-5 中“标准名称 + 别名”匹配能力存在；这是本次评审明确要求验证的回归保护，不能仅依赖实现代码的静态正确性。 + +要求的修复方向： + +- 增加独立技能/独立场景，确保某个标准技能只能通过 `nameZh` 命中，另一个只能通过 alias 命中。 +- 保留现有 `nameEn` 和 `skillCode` 隔离断言。 +- 建议再断言 alias 必须归属于目标 `skillId`，避免别名被错误应用于其他标准技能时测试仍通过。 + +## 已核验且未发现阻断的行为 + +- 匹配集合直接对 `SkillCloudQueryService.query(...)` 的结果取 `subList(0, min(10, size))`，服务内未重新排序。 +- 无 profile、空白 JSON、空数组和语法损坏 JSON 均按无技能处理；损坏 JSON 测试验证不抛 500 且日志不含原始 JSON。 +- 归一化覆盖 NFKC、trim、`Locale.ROOT` 小写、Unicode 连字符、空白折叠、句点移除，并保留 `+`、`#`。 +- `nameEn` 和 `skillCode` 在现有测试中有可区分断言；归一化后的学生技能使用 `Set` 去重。 +- 分数使用 `Math.round(matched * 100.0 / actualTopSize)`，`actualTopSize=0` 返回 0，`1/3=33`，少于十条和恰好十条均有断言。 +- 命中/缺失列表按 TOP 顺序构建，并无额外排序；无匹配场景的并集顺序与词云 TOP10 对齐。 +- 响应 DTO 只包含技能投影，不包含原始 `skills_json`、手机号、邮箱或简历内容。 +- T6 为只读事务，无持久化；未实现 T7/T8。 + +## 未决问题 + +- 无设计或接口语义待裁决。 +- 待修复 T6 测试可证伪性后，必须重新执行同一回归命令并进行独立复审；复审 `PASS` 前不得启动 T7。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T7-rereview.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T7-rereview.md new file mode 100644 index 0000000..5264be0 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T7-rereview.md @@ -0,0 +1,80 @@ +# W2-T7 顺序测试修复复审报告 + +## Verdict + +**PASS** + +- 复审对象：`mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillLearningPathServiceTest.java` +- 对照报告：`changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T7.md` +- 生产服务基线：`fca1160 feat: add skill learning path service` +- 结论：T7-I1 已关闭；当前顺序测试可证伪，原有 T7 行为测试未回归，未发现 T8 实现。 + +## Finding 关闭状态 + +### T7-I1：测试无法阻止学习路径按名称重排 + +**已关闭。** + +1. Mockito 场景的输入名称和 ID 均不等于自然升序： + - `SkillLearningPathServiceTest.java:135-139` + - 名称输入为 `Skill Z -> Skill A -> Skill M`，自然升序为 `Skill A -> Skill M -> Skill Z`。 + - ID 输入为 `30 -> 10 -> 20`，自然升序为 `10 -> 20 -> 30`。 + - `SkillLearningPathServiceTest.java:155-156` 显式断言输入名称、ID 都不等于自然排序结果。 + - `SkillLearningPathServiceTest.java:157-165` 读取服务返回对象，并严格断言返回 ID 为 `30, 10, 20`、名称为 `Skill Z, Skill A, Skill M`。 + - 因此，生产实现若按名称或 ID 重排，现有测试必然失败。 + +2. 断言读取的是服务返回结果，而不是只验证 Mockito 输入： + - `SkillLearningPathServiceTest.java:157-165` 使用 `response.items()` 的 `skillId` 和 `skillName` 做精确顺序断言。 + - `SkillLearningPathServiceTest.java:160-162` 同时断言优先级严格为 `1, 2, 3`。 + - 满足“返回顺序、priority 和技能 ID”均被验证的要求。 + +3. 新增 V36 真实数据场景覆盖不同热度分组和名称顺序： + - `SkillLearningPathServiceTest.java:89-121` + - 使用教育行业 `3`、教学设计职位 `9` 调用真实 `SkillMatchService` 和 `SkillLearningPathService`。 + - `SkillLearningPathServiceTest.java:102-110` 断言缺失技能共 10 项、存在多个不同热度值，且名称和 ID 顺序均不等于自然升序。 + - `SkillLearningPathServiceTest.java:112-120` 将服务返回的 `skillId`、`skillName` 与 `matchResponse.missingSkills()` 的顺序做严格比较，并断言优先级为 `1..10`。 + - 因此，真实数据场景不是仅验证输入或数量，而是验证了学习路径严格复用匹配结果的缺失技能顺序。 + +4. 原有 T7 行为测试未回归： + - 四类模板测试：`SkillLearningPathServiceTest.java:176-265`，包括编程语言、框架、工具、通用能力、未知/空模板及分类不存在回退。 + - 空路径测试：`SkillLearningPathServiceTest.java:268-280`，断言 `total=0`、空列表且不访问分类仓储。 + - 重复调用测试：`SkillLearningPathServiceTest.java:282-309`，断言顺序、优先级和文本一致。 + - 本次修复仅修改测试文件；`SkillLearningPathService.java` 相对 `fca1160` 的生产差异为空。 + +5. 未发现 T8 实现： + - 当前 HEAD 仍为 `fca1160`。 + - 源码和测试目录未发现 `SkillCloudController`、`SkillLearningPathController`、`/skills` 词云或学习路径接口及对应安全配置变更。 + - 未发现超出 T7 范围的生产实现。 + +## 独立测试结果 + +环境：JDK 17.0.16、Maven 3.9.7。 + +执行命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest' test +``` + +结果： + +```text +Tests run: 50, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +分类统计： + +```text +SkillTaxonomyMigrationTest: 2 passed +SkillCloudSeedDataIntegrationTest: 8 passed +SkillTaxonomyRepositoryTest: 6 passed +SkillCloudQueryServiceTest: 8 passed +SkillMatchServiceTest: 16 passed +SkillLearningPathServiceTest: 10 passed +``` + +## 剩余阻断项 + +无。T7-I1 已关闭，允许进入 T8 评审前置流程。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T7.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T7.md new file mode 100644 index 0000000..67fd2ee --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-T7.md @@ -0,0 +1,72 @@ +# W2-T7 独立评审报告 + +## Verdict + +**FAIL** + +- 提交：`fca1160 feat: add skill learning path service` +- 评审对象：`SkillLearningPathService`、`SkillLearningPathItem`、`SkillLearningPathResponse`、`SkillLearningPathServiceTest` +- 阻断项：1 项 Important +- 结论：生产实现整体符合 T7 行为契约，但核心顺序行为缺少可证伪的回归测试，不能放行 T8。 + +## 独立测试 + +执行环境：JDK 17.0.16、Maven 3.9.7。 + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest' test +``` + +结果： + +```text +Tests run: 49, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 1:00 min +``` + +## Important + +### T7-I1：测试无法阻止学习路径按名称重排 + +- 文件：`mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillLearningPathServiceTest.java:98-117` +- 相关实现：`mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillLearningPathService.java:35-38` + +现有单测的 `missing` 顺序为： + +- `First` / `skillId=31` / `heat=1.00` +- `Second` / `skillId=32` / `heat=10.00` +- `Third` / `skillId=33` / `heat=5.00` + +期望输出保持该输入顺序。但输入的名称和 `skillId` 本身均为升序，因此把实现改成： + +```java +missingSkills.stream() + .sorted(Comparator.comparing(SkillCloudItem::nameZh)) + .map(...) +``` + +后，单测仍会通过。真实数据集成测试 `SkillLearningPathServiceTest.java:62-76` 使用职位 1，其当前 TOP10 全部为相同热度，名称顺序也已符合词云次级排序，因此同样无法发现该重排缺陷。 + +T7 的要求是严格复用 `SkillMatchService` 返回的 `missingSkills` 顺序，不自行排序；该回归会破坏 REQ-LP-2。测试需要使用不会被名称排序或 `skillId` 排序偶然保持一致的缺失技能顺序，最好增加多热度分组的真实数据场景。 + +## 已通过检查 + +- 学习路径只遍历 `matchResponse.missingSkills()`，未消费已命中技能；优先级从 1 连续生成。 +- 编程语言、框架、工具、通用四类模板均已实现；未知、空模板编码及分类不存在时回退通用模板。 +- 建议包含技能名，精确断言中未出现 URL、课程链接、讲师、价格、时长、完成度或 AI 内容。 +- 无缺口时返回 `total=0` 和空列表，且不访问分类仓储。 +- 重复调用返回相同顺序、优先级和建议。 +- `SkillLearningPathItem` 仅包含技能 ID、技能名、分类 ID、分类名、优先级和建议，不含隐私字段。 +- 服务使用只读事务；未发现持久化或外部服务调用。 +- 提交只新增 4 个指定文件，未包含 T8 及后续实现。 + +## Minor + +- `SkillLearningPathServiceTest.java:195-221` 覆盖了 null 模板编码、未知模板和分类不存在，但未直接覆盖 `skill.categoryId()==null`；实现已在 `SkillLearningPathService.java:57-58` 回退 GENERAL。 +- 外部内容主要通过精确模板断言和禁用词断言约束；生产类没有外部客户端依赖，因此本项不构成阻断。 + +## 未决问题 + +无。修复测试后需重新执行指定回归并复审。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-wave.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-wave.md new file mode 100644 index 0000000..d723c9f --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W2-wave.md @@ -0,0 +1,175 @@ +# N03 SDD W2 整批独立评审报告 + +## 评审对象 + +- 范围：`b960036..f296df4` +- Wave：W2 +- 批次：B2 / T4–T7 +- 提交： + - `0c77824 feat: add skill taxonomy repositories` + - `52751e5 fix: constrain skill projections by occupation category` + - `e3eecfe feat: add skill cloud query service` + - `1666162 fix: expose skill source metadata in cloud items` + - `6eb30e7 feat: add personal skill match service` + - `fca1160 feat: add skill learning path service` + - `f296df4 test: harden learning path ordering regression` +- 评审依据：修订后的 `execution-contract.md`、`design.md`、`tasks.md`、三份 spec、V35/V36，以及 `W2-T4*`、`W2-T5*`、`W2-T6*`、`W2-T7*` 报告。 +- 环境：`JAVA_HOME=D:\java\jdk`，Maven `D:\maven\apache-maven-3.9.7\bin\mvn.cmd`，工作目录 `D:\1\zhi\zhilai\mydreamjob-server`。 + +## Verdict + +**PASS** + +W2/T4–T7 的领域与查询实现完整覆盖 B2 范围；词云、个人匹配、学习路径确实共享同一排名管道；未发现 Critical 或 Important 阻断项。W3/T8 可以开始，但必须先处理报告中的 Minor 项，并在 B3 补齐 HTTP、权限、响应序列化、隐私和真实 MySQL 验证。 + +## 测试结果 + +独立执行： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest' test +``` + +结果： + +```text +Tests run: 50, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +Surefire 分类： + +| 测试类 | 通过数 | +| --- | ---: | +| `SkillTaxonomyMigrationTest` | 2 | +| `SkillCloudSeedDataIntegrationTest` | 8 | +| `SkillTaxonomyRepositoryTest` | 6 | +| `SkillCloudQueryServiceTest` | 8 | +| `SkillMatchServiceTest` | 16 | +| `SkillLearningPathServiceTest` | 10 | + +补充静态检查： + +- `git diff --check b960036..f296df4` 通过。 +- `git log --oneline b960036..f296df4` 只包含上述 7 个 W2/T4–T7 提交。 +- 范围内实现文件集中在 `mydreamjob-server/src/main/java/com/mydreamjob/skill/**` 与对应测试；未发现 `SkillCloudController`、`SecurityConfig` 或 OpenAPI 变更。 + +## 实现与需求覆盖矩阵 + +| 需求 | W2 结论 | 主要实现与证据 | +| --- | --- | --- | +| REQ-SC-1 | 不属于 W2，待 B3 | 未实现 Controller/SecurityConfig；由 T8 负责。W2 只提供可被 Controller 调用的服务。 | +| REQ-SC-2 | 通过（服务层） | `SkillCloudQueryService.query` 校验必填、正整数、行业—职位关系、分类—职位关系，非法组合抛 `BusinessException(BAD_REQUEST)`，见 `SkillCloudQueryService.java:57-79`、`150-158`；测试见 `SkillCloudQueryServiceTest.java:238-264`。 | +| REQ-SC-3 | 通过 | `heat DESC, nameZh ASC, skillId ASC`，`null` heat 末位，见 `SkillCloudQueryService.java:28-40`、`86-99`、`120-144`；测试覆盖空 heat、同 heat、同名称 skillId，见 `SkillCloudQueryServiceTest.java:84-145`。 | +| REQ-SC-4 | 通过（B2） | 词云返回完整列表；匹配直接取 `min(10, actualSize)`，未另建排序，见 `SkillMatchService.java:53-55`；测试见 `SkillMatchServiceTest.java:85-118`。 | +| REQ-SC-5 | B2 通过，B3 待补 | B2 已返回 rank、技能身份、分类、heat、技能来源及热度来源字段，见 `SkillCloudItem.java:5-19`、`SkillCloudQueryService.java:102-117`；`ApiResponse` 包装属于 T8。 | +| REQ-SM-1 | 通过 | 匹配服务复用 `SkillCloudQueryService.query` 的结果前 `min(10, actualSize)` 项，见 `SkillMatchService.java:53-55`；测试见 `SkillMatchServiceTest.java:85-137`。 | +| REQ-SM-2 | B2 通过，B3 待补 | 服务只通过 `studentUserId` 查询 `student_profiles.skills_json`，缺失、空、损坏 JSON 均按空技能处理，见 `SkillMatchService.java:81-116`；JWT 身份提取必须由 T8 Controller 完成。 | +| REQ-SM-3 | 通过 | NFKC、trim、Locale.ROOT、Unicode 连字符、连续空白、句点、`+`/`#` 规则见 `SkillNameNormalizer.java:12-27`；nameZh/nameEn/skillCode/alias 和别名归属有隔离测试，见 `SkillMatchServiceTest.java:237-292`。 | +| REQ-SM-4 | 通过 | `round(matched * 100.0 / actualTopSize)`，空集合返回 0，重复学生技能通过 `Set` 去重，见 `SkillMatchService.java:54-77`、`100-110`；测试覆盖 1/3=33、全命中、空集合。 | +| REQ-SM-5 | 通过 | 命中/缺失列表按 TOP 顺序分别构造，并集等于实际 TOP10，见 `SkillMatchService.java:59-78`；测试见 `SkillMatchServiceTest.java:294-370`。 | +| REQ-LP-1 | 通过 | 学习路径完全由 `SkillMatchResponse.missingSkills()` 驱动，不重新加入已命中或 TOP10 外技能，见 `SkillLearningPathService.java:29-40`；测试见 `SkillLearningPathServiceTest.java:131-174`。 | +| REQ-LP-2 | 通过 | 直接沿用 missingSkills 顺序，优先级从 1 连续递增，见 `SkillLearningPathService.java:35-38`；真实 V36 多热度分组及名称/ID 乱序测试见 `SkillLearningPathServiceTest.java:88-121`。 | +| REQ-LP-3 | 通过 | 四类确定性模板及未知模板回退见 `SkillLearningPathService.java:56-93`；模板精确断言见 `SkillLearningPathServiceTest.java:176-265`。 | +| REQ-LP-4 | 通过（B2） | 路径项包含技能 ID、名称、分类、优先级和建议，建议文本不含链接、课程、价格、时长、完成度或 AI 内容，见 `SkillLearningPathItem.java:3-10`、`SkillLearningPathServiceTest.java:358-362`。 | + +## 关键检查结论 + +### 1. 三服务共享同一排名管道：通过 + +`SkillCloudQueryService` 是唯一排序实现： + +- 过滤：`SkillCloudQueryService.java:57-84` +- 排序：`SkillCloudQueryService.java:28-40`、`86-88` +- 排名：`SkillCloudQueryService.java:89-91` +- 来源映射：`SkillCloudQueryService.java:102-117` + +`SkillMatchService` 直接调用词云服务并截取前 `min(10, actualSize)` 项，没有复制排序逻辑，见 `SkillMatchService.java:53-55`。`SkillLearningPathService` 直接消费匹配服务的 `missingSkills`，没有重新排序，见 `SkillLearningPathService.java:33-38`。因此 TOP10、匹配缺口和学习路径的顺序语义一致。 + +### 2. 跨任务接口一致性：通过，存在命名待收口项 + +DTO 字段和顺序语义互相衔接： + +- `SkillCloudItem` 提供词云和匹配共用的完整技能项，见 `SkillCloudItem.java:5-19`。 +- `SkillMatchResponse` 返回 `matchScore`、`matchedCount`、`totalCount`、`matchedSkills`、`missingSkills`，见 `SkillMatchResponse.java:5-15`。 +- `SkillLearningPathResponse` 返回 `total`、`items`；`SkillLearningPathItem` 返回 `priority`、技能、分类和建议，见 `SkillLearningPathResponse.java:5-11`、`SkillLearningPathItem.java:3-10`。 + +Minor M1：设计文档 `design.md:131` 使用概念字段名 `heatSource`，而实现采用扁平字段 `heatSourceCode`、`heatElementId`、`heatProxy`，见 `SkillCloudItem.java:15-18`。这不是 B2 行为错误，但 T8 必须在 OpenAPI、接口文档和前端契约中明确最终 JSON 命名，避免文档与运行时漂移。 + +### 3. 领域边界、事务、空值和异常：通过 + +- 三个业务服务均为 `@Transactional(readOnly = true)`，没有写操作，见三个服务类的类级注解。 +- 分类—职位约束同时存在于服务校验和 Repository 投影查询中；跨职位分类错配测试见 `SkillTaxonomyRepositoryTest.java:123-152`。 +- 空 heat 保留并末位排序：`SkillCloudQueryService.java:120-131`。 +- 空结果：词云返回 `total=0, items=[]`；匹配返回 `matchScore=0`、空命中/缺失；学习路径返回空列表，见对应测试。 +- 损坏 JSON 被捕获并仅记录异常类型和 profile ID，不输出原始 JSON，见 `SkillMatchService.java:91-114`。 +- 学习路径未知/空/缺失模板回退通用模板，见 `SkillLearningPathService.java:56-65`、`84-93`。 + +### 4. 确定性与性能：通过，Minor N+1 + +- 排序使用 `BigDecimal.compareTo`、`nameZh`、`skillId` 全序比较，重复请求结果确定。 +- 每个缺失技能最多一次分类模板查询，构建学习路径时最多 10 次查询，见 `SkillLearningPathService.java:61-64`。 + +Minor M2：这是有界的 N+1。当前任务是代表性小词库、最多 10 个路径项，不构成 W2 阻断；如果后续词库或 TOP 上限扩大，应在 T8/T10 或后续优化中改为一次批量读取模板。 + +### 5. 测试可证伪性：通过，有少量 Minor 缺口 + +已覆盖用户要求的关键反例： + +- 跨职位分类错配：`SkillTaxonomyRepositoryTest.java:123-152`、`SkillMatchServiceTest.java:347-370`。 +- 排序错误：`SkillCloudQueryServiceTest.java:84-145` 覆盖热度、同名/同热度的 skillId 稳定排序。 +- 名称、别名和别名归属遗漏：`SkillMatchServiceTest.java:237-292`。 +- 分母错误：`SkillMatchServiceTest.java:120-137`。 +- 优先级重排：`SkillLearningPathServiceTest.java:88-174`。 +- 模板遗漏：`SkillLearningPathServiceTest.java:176-265`。 +- 空路径：`SkillLearningPathServiceTest.java:268-280`。 + +Minor M3：B2 测试未直接覆盖 `SkillLearningPathService.resolveTemplate` 在 `skill.categoryId()==null` 时的通用模板分支；也未通过完整序列化验证所有响应对象不含原始 `skills_json`。生产 DTO 当前不含隐私字段，且 `SkillLearningPathServiceTest.java:358-362` 已约束建议文本；T8/T9 的 MockMvc/端到端测试必须补齐响应级隐私断言。 + +### 6. W2 未越过 T8 边界：通过 + +`git diff --name-only b960036..f296df4` 未出现： + +- `SkillCloudController` 或任何 `/skills/**` Controller； +- `SecurityConfig` 变更； +- OpenAPI 文件； +- `/api/v1/skills/*` 运行时端点。 + +因此 W2 没有实现 T8 的接口、权限或 OpenAPI 工作，边界符合 `tasks.md:24-25` 的后续安排。 + +## 严重级别与发现 + +### Critical + +无。 + +### Important + +无。 + +### Minor + +- **M1：设计中的 `heatSource` 与实际扁平来源字段未最终对齐。** + - 位置：`changes/n03-skill-cloud/design.md:131`；`mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudItem.java:15-18`。 + - 处理方向：T8 明确选择扁平 `heatSourceCode`/`heatElementId`/`heatProxy` 或嵌套 `heatSource`，并同步 OpenAPI、接口文档和前端示例。 + +- **M2：学习路径模板查询是有界 N+1。** + - 位置：`mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillLearningPathService.java:61-64`。 + - 处理方向：当前最多 10 次查询，不阻断；若词库或 TOP 上限扩大，应批量读取 `(categoryId, occupationId)` 模板映射。 + +- **M3：B2 测试未直接覆盖空 `categoryId` 模板回退和完整响应序列化隐私断言。** + - 位置：`mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillLearningPathServiceTest.java:240-265`、`358-362`。 + - 处理方向：T8 Controller 测试必须验证三个接口的响应不含 `skills_json`、手机号、邮箱和简历内容；可同时补 `categoryId=null` 的 Service 单测。 + +## 未决问题 + +1. `SkillCloudItem` 热度来源字段的最终 JSON 形态尚未在设计中收口，需在 T8 同时对齐实现、OpenAPI 和前端联调文档。 +2. B2 只验证 H2 `MODE=MySQL` 与 Service 层异常类型；真实 MySQL 迁移、HTTP 401/403/400、JWT 到 `studentUserId` 的绑定和 `ApiResponse` 包装仍属于 B3。 +3. B2 没有执行 `mvn clean verify` 或静态 OpenAPI 校验；这些属于 T9/T10 的退出条件，当前不作为 W2 阻断。 + +## 是否放行 W3 + +**放行。** + +W2 的领域行为、排名管道、个人匹配、学习路径和可证伪测试已达到 B2 完成定义。允许进入 W3/T8；W3 开始前必须在实现 Controller/SecurityConfig/OpenAPI 时处理 M1，并在 T8–T10 验证上述 HTTP、权限、隐私和真实数据库边界。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T8-rereview.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T8-rereview.md new file mode 100644 index 0000000..00a4409 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T8-rereview.md @@ -0,0 +1,105 @@ +# W3-T8 独立复审报告 + +## 复审结论 + +- Verdict：**PASS** +- 复审范围： + - `mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudController.java` + - `mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudControllerTest.java` + - `mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudAuthorizationIntegrationTest.java` + - 对照前次报告 `changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T8.md` +- 复审方式：静态核对实现、测试断言、DTO、JWT/角色链路与测试清理；独立运行指定 8 个测试类。未修改任何实现或测试文件。 + +## 原 Important finding 关闭状态 + +### T8-I1：CLOSED + +前次问题为三个 OpenAPI 200 示例的 `total`、计数与数组长度矛盾，且测试未读取示例内容。当前修复已关闭该问题： + +- `SkillCloudController.java:56`：词云示例改为 `total=1` 且 `items` 为 1 项。 +- `SkillCloudController.java:103`：匹配示例改为 `matchScore=50`、`matchedCount=1`、`totalCount=2`，命中 1 项、缺失 1 项。 +- `SkillCloudController.java:150`：学习路径示例改为 `total=1` 且 `items` 为 1 项。 +- `SkillCloudControllerTest.java:267-307`：测试读取 `/v3/api-docs` 中的 `responses.200.content.application/json.examples.success.value`，使用 Jackson 解析字符串示例或对象示例。 +- `SkillCloudControllerTest.java:278-280`：词云 data 字段集合精确匹配，`total == items.size()`，技能项字段集合与 `SkillCloudItem` 一致。 +- `SkillCloudControllerTest.java:282-292`：匹配字段集合精确匹配，`matchedSkills.size == matchedCount`、`missingSkills.size == totalCount - matchedCount`，并按计数校验 `matchScore`。 +- `SkillCloudControllerTest.java:294-299`：学习路径字段集合精确匹配，`total == items.size()`，路径项字段集合与 `SkillLearningPathItem` 一致。 +- 对照 DTO： + - `SkillCloudResponse.java`：`industry`、`occupation`、`category`、`total`、`items`。 + - `SkillCloudItem.java`：`rank`、`skillId`、`skillCode`、`nameZh`、`nameEn`、`sourceType`、`sourceCode`、`categoryId`、`categoryName`、`heat`、`heatSourceCode`、`heatElementId`、`heatProxy`。 + - `SkillMatchResponse.java`：`matchScore`、`matchedCount`、`totalCount`、`matchedSkills`、`missingSkills`。 + - `SkillLearningPathResponse.java`：`total`、`items`。 + - `SkillLearningPathItem.java`：`priority`、`skillId`、`skillName`、`categoryId`、`categoryName`、`suggestion`。 + +可证伪性：单独篡改 `matchScore` 会在 `SkillCloudControllerTest.java:289-290` 失败；篡改 `total` 会在 `:279`、`:288` 或 `:297` 失败；缩短或扩长数组会因元素数量不一致失败；直接删除示例或改坏 JSON 会在 `:302-306` 失败。 + +### T8-I2：CLOSED + +前次问题为角色测试通过 Mock `ProfileService` 按 `userId` 决定结果，未证明真实数据库角色集合。当前修复已关闭该问题： + +- `SkillCloudAuthorizationIntegrationTest.java:42-46`：使用 `@SpringBootTest`、MockMvc、真实 `integration-test` profile 和独立 H2 数据库。 +- `:55-74`：测试注入真实 `UserRepository`、真实 `JwtTokenProvider`；只 Mock 三个技能服务及 Redis，不 Mock `ProfileService`。 +- `:95-108`：真实保存 `STUDENT`、`COMPANY`、`ADMIN`、`STUDENT+COMPANY`、`COMPANY+ADMIN` 五种角色集合，并在三个端点上分别断言 200、403、403、200、403。 +- `SkillCloudController.java:185-198`：身份只取认证主体 `userId`，随后调用真实 `profileService.assertStudentRole(userId)`。 +- `ProfileService.java:50-57`：真实读取 `UserRepository` 并按 `User.roles` 中是否包含 `STUDENT` 判断。 +- `JwtAuthenticationFilter.java:34-40`：JWT 仅用于解析 subject 并建立认证主体；测试 token 即使统一携带 `roles=STUDENT` claim，生产链路仍按数据库角色拒绝 COMPANY/ADMIN，因此断言真实可证。 + +可证伪性：若实现改为信任 JWT 的 `roles` claim，COMPANY/ADMIN 场景会因测试 token 的 `STUDENT` claim 返回 200 而失败；若改回 Mock 按 userId 放行，数据库角色集合测试在本类中无法满足，且真实 `ProfileService` 未替换。 + +## 额外身份查询参数 + +- `SkillCloudAuthorizationIntegrationTest.java:112-130`：对三个端点同时传入公司用户的 `userId` 和 `studentId`，但使用学生 JWT；全部返回 200，并验证 `SkillMatchService` 与 `SkillLearningPathService` 收到 JWT 学生 ID，`SkillCloudQueryService` 只收到筛选参数。 +- `SkillCloudController.java:73-83`、`:120-130`、`:167-177`：三个接口均未声明 `userId` 或 `studentId` 请求参数，身份只来自 `resolveAuthenticatedStudent()`。 +- 结论：额外查询参数不能覆盖 JWT 主体。 + +## 端点、参数与安全回归 + +- `SkillCloudController.java:73`、`:120`、`:167`：三个 GET 路由保持为 `/skills/word-cloud`、`/skills/match`、`/skills/learning-path`。 +- 三个方法均要求正整数 `industryId`、`occupationId`，可选正整数 `categoryId`；`@Validated` 与 `@Positive` 生效。 +- `SecurityConfig.java:77-78`：`GET /skills/**` 要求 authenticated，匿名请求无法进入 Controller。 +- `SkillCloudControllerTest.java:156-165`：三个端点匿名请求均断言 401。 +- `SkillCloudControllerTest.java:168-206`：覆盖必填参数缺失、非正整数和可选参数非法返回 400。 +- `SkillCloudControllerTest.java:110-154`：三个 STUDENT 成功路径均返回统一 `ApiResponse` 和对应 Service DTO 载荷。 +- 未发现 T8 修改导致三个端点、参数校验或 SecurityConfig 行为回归。 + +## 测试清理与范围 + +- `SkillCloudControllerTest.java:50-51` 与 `SkillCloudAuthorizationIntegrationTest.java:45-46` 使用不同的 H2 数据库名，避免类别间共享数据。 +- `SkillCloudControllerTest.java:105-108`、`SkillCloudAuthorizationIntegrationTest.java:89-92` 均在每个测试后执行 `userRepository.deleteAll()`。 +- 指定 8 个测试类完成后全部通过，未观察到外键残留或后续测试污染。 +- 当前工作区中未出现 `docs/skill-cloud-api.md`，未修改 `docs/openapi-rc.json`、`docs/frontend-integration.md`，未实现 T9/T10。 + +## 独立测试结果 + +环境：JDK 17.0.16，Maven 3.9.7。 + +执行命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest,SkillCloudControllerTest,SkillCloudAuthorizationIntegrationTest' test +``` + +结果： + +```text +Tests run: 66, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 53.008 s +``` + +分类结果： + +- `SkillTaxonomyMigrationTest`：2 +- `SkillCloudSeedDataIntegrationTest`：8 +- `SkillTaxonomyRepositoryTest`：6 +- `SkillCloudQueryServiceTest`：8 +- `SkillMatchServiceTest`：16 +- `SkillLearningPathServiceTest`：10 +- `SkillCloudControllerTest`：14 +- `SkillCloudAuthorizationIntegrationTest`：2 + +## 剩余阻断项 + +无 Critical、Important 阻断项。T8-I1、T8-I2 均已关闭，T8 可进入后续流程。 + +非阻断备注：200 响应目前通过示例和测试断言锁定字段及计数，未显式绑定具体响应 schema；本轮要求已满足，不阻断 T8。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T8.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T8.md new file mode 100644 index 0000000..1df8931 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T8.md @@ -0,0 +1,87 @@ +# W3-T8 独立评审报告 + +## 评审结论 + +- Verdict：**FAIL** +- 评审对象： + - `mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudController.java` + - `mydreamjob-server/src/main/java/com/mydreamjob/config/SecurityConfig.java` + - `mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudControllerTest.java` +- 依据：修订后的 `execution-contract.md`、`design.md`、`tasks.md`，三份 spec，以及现有 JWT/Security/Controller 约定。 +- 评审方式：静态核对实现与契约，独立运行指定回归，检查 Git 工作区范围；未修改实现文件。 + +## 独立测试结果 + +执行命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest,SkillCloudControllerTest' test +``` + +结果： + +```text +Tests run: 69, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +``` + +其中 `SkillCloudControllerTest` 为 19 项通过。`git diff --check HEAD` 对 T8 三个文件无空白错误。`docs/openapi-rc.json`、`docs/frontend-integration.md` 无改动，`docs/skill-cloud-api.md` 不存在；未发现 T9/T10 实现。 + +## 阻断项 + +### T8-I1（Important）：OpenAPI 成功示例与 DTO/运行时契约矛盾 + +文件与行号： + +- `SkillCloudController.java:56`：词云示例中 `data.total=15`，但 `items` 只有 1 项。 +- `SkillCloudController.java:103`：匹配示例中 `matchedCount=8`、`totalCount=10`，但同时返回 `matchedSkills=[]`、`missingSkills=[]`。 +- `SkillCloudController.java:150`：学习路径示例中 `data.total=2`，但 `items` 只有 1 项。 +- `SkillCloudControllerTest.java:247-269`：OpenAPI 测试只检查路径存在、`bearerAuth`、状态码和参数必填性，没有校验示例内容或 200 响应 schema，因此上述矛盾不会使测试失败。 +- `changes/n03-skill-cloud/execution-contract.md:120`：明确要求“OpenAPI 注解、示例和实际响应必须一致”。 + +影响：客户端联调、接口文档和运行时契约可能按错误示例理解 TOP10、命中/缺失列表与总数关系。应修正三个示例，使计数、数组长度和 DTO 字段一致，并在 T8 的 OpenAPI 测试中锁定关键示例字段；同时建议为 200 响应声明对应 `SkillCloudResponse`、`SkillMatchResponse`、`SkillLearningPathResponse` schema，而非只有示例。 + +### T8-I2（Important）：角色测试通过 Mock 按 userId 放行，未证明真实 STUDENT 校验和多角色集合 + +文件与行号： + +- `SkillCloudControllerTest.java:93-97`：`ProfileService` 被 `@MockBean` 替换，并按 `100L/200L/300L` 直接设置成功或抛 `FORBIDDEN`。 +- `SkillCloudControllerTest.java:161-171`：测试传入 `COMPANY`/`ADMIN` 角色声明，但实际 403 取决于 mock 对 `200L` 的硬编码行为，而非数据库角色集合。 +- `SkillCloudControllerTest.java:296-301`：JWT 的 `roles` claim 只作为测试辅助值传入。 +- `SkillCloudController.java:185-198`：Controller 确实从 `SecurityContext` 取身份并调用 `profileService.assertStudentRole(userId)`。 +- `ProfileService.java:50-57`：生产实现从数据库读取用户，并使用 `anyMatch(STUDENT)`，因此单角色和包含 STUDENT 的多角色在代码层面是正确的。 +- 现有 `ProfileControllerTest.java:167-185` 覆盖了数据库单角色 COMPANY/ADMIN 的 403，但没有覆盖 STUDENT+COMPANY 或 STUDENT+ADMIN 多角色集合，也没有覆盖本组 `/skills/**` 端点的真实角色链路。 + +影响：T8 的 Controller 测试无法证明 `COMPANY/ADMIN` 是被真实服务端角色检查拒绝，也无法证明多角色用户中只要包含 STUDENT 就能通过；它可能在 `ProfileService.assertStudentRole` 被替换或角色逻辑回归时仍通过。应增加使用真实 `ProfileService`、真实数据库用户的集成场景，至少覆盖： + +- COMPANY、ADMIN：预期 403。 +- STUDENT+COMPANY：预期允许。 +- 无 STUDENT 的多角色集合：预期 403。 + +## 非阻断项 + +### T8-M1：身份注入测试只覆盖 `/skills/match` + +文件与行号：`SkillCloudControllerTest.java:214-225`。 + +当前实现三个端点都调用同一个 `resolveAuthenticatedStudent()`，代码层面正确；但测试只验证 `match` 端点会忽略 `userId`/`studentId`，没有对词云和学习路径端点做同样断言。建议参数化覆盖三个路径，并 `verify` 服务收到的是 JWT 中的 `100L`。 + +### T8-M2：隐私断言只匹配 `skills_json` + +文件与行号：`SkillCloudControllerTest.java:227-244`。 + +测试没有覆盖驼峰形式 `skillsJson` 或其他可能的新增个人技能字段。当前返回 DTO 为受控记录类型，因此非阻断；建议后续端到端测试使用真实服务响应，并对 DTO schema/序列化字段做更严格断言。 + +## 检查结论 + +- 三个路径和参数语义：**通过**。`industryId`、`occupationId` 必填，`categoryId` 可选；实测缺参/0/负数返回 400。 +- 身份来源：**通过**。Controller 只从 `SecurityContext` 取主体，不绑定 `userId`/`studentId`。 +- 权限行为：**实现通过，测试证明不完整**。匿名 401、COMPANY/ADMIN 403 的 MockMvc 场景通过；但角色拒绝依赖 mock，多角色未覆盖。 +- `ApiResponse` 与成功 DTO：**通过**。成功响应统一包装，字段与 Service DTO 一致。 +- OpenAPI：**不通过**。注解具备 Tag、Operation、SecurityRequirement、参数说明和 200/400/401/403，但成功示例与 DTO/运行时数据矛盾，且 200 响应缺少明确 schema。 +- 改动范围：**通过**。未修改 `docs/openapi-rc.json`，未实现 T9/T10。 + +## 未决问题 + +- 当前安全层的 401/403 JSON 由既有 `SecurityConfig` 直接写出，不是 `ApiResponse` 实体序列化结果；是否在本变更中统一该历史约定，需由项目维护者决定。本次未将其列为 T8 阻断项，因为它与仓库既有安全配置约定一致。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T9.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T9.md new file mode 100644 index 0000000..c9bea9a --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-T9.md @@ -0,0 +1,177 @@ +# W3-T9 独立评审报告 + +## 评审结论 + +- Verdict：**PASS** +- T9 是否允许提交：**允许** +- 是否允许继续 T10：**允许继续实现与文档阶段，但不得在本次全量回归未全绿前判定 T10/B3 验证通过** +- 评审范围： + - `mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudEndToEndIntegrationTest.java` + - `target/skill-cloud-t9-red.log` + - `target/skill-cloud-t9-green.log` + - `target/skill-cloud-t2-t9-regression.log` + - `target/skill-cloud-t9-full-regression.log` + - `target/skill-cloud-t9-unrelated-failures-isolated.log` + - 修订后的 contract、design、tasks、三份 spec，以及 T8 Controller +- 评审方式：静态核对测试与生产链路，使用 JDK 17 / Maven 3.9.7 独立执行指定 T2–T9 测试组、四个失败测试类及完整 `mvn test`。未修改实现、测试或其他源码文件。 + +## T9 独立结论 + +### 真实链路 + +T9 使用 `@SpringBootTest`、`@AutoConfigureMockMvc`、`@ActiveProfiles("integration-test")` 和独立 H2 数据库，实际经过 Controller、Security、ProfileService、Skill 查询/匹配/学习路径 Service、Repository、Flyway 与 V36 种子数据。仅 `RedisTemplate` 被 Mock，核心业务链路没有被替换。 + +可证伪证据： + +- `SkillCloudEndToEndIntegrationTest.java:48-55`：真实 Spring 上下文、MockMvc、H2 和生产数据库 URL/Profile。 +- `:71-90`：注入真实 MockMvc、JdbcTemplate、JwtTokenProvider、Repository；唯一 `@MockBean` 为 RedisTemplate。 +- `:100-134`：真实 STUDENT JWT 和真实 StudentProfile 调用三个端点，断言后端职位返回 16 项真实 V36 技能、rank 连续、heat 降序。 +- `:269-304`：匹配 union、命中/缺失顺序、学习路径 priority/顺序、建议非空均有断言。 +- 无 Flyway 或种子数据时，依赖的固定节点、16 项技能和 V36 来源字段无法成立；测试已独立通过，证明不是纯 Mock 测试。 + +### 词云 TOP10 与匹配一致性 + +T9 从真实词云结果直接取 `allSkills.subList(0, min(10, allSkills.size()))` 作为 TOP10，再验证： + +- `totalCount` 等于实际 TOP10 大小。 +- `matchedCount` 等于 matchedSkills 大小。 +- matchedSkills 与 missingSkills 的并集严格等于词云 TOP10。 +- matchedSkills 保持词云顺序，missingSkills 保持词云顺序。 +- `matchScore` 等于 `round(matchedCount * 100 / expectedTop10.size())`。 +- 3 项技能场景验证 `1/3 = 33`，并验证空热度末位。 + +该设计可证伪“匹配服务另起排序”“TOP10 与实际列表不一致”“missingSkills 丢项或重排”等缺陷。 + +### 学习路径 + +T9 将学习路径逐项与 `match.missingSkills` 对比： + +- 总数为缺失技能数。 +- 每项 `priority` 从 1 连续递增。 +- `skillId`、`skillName` 与缺失技能逐项一致。 +- `suggestion` 非空。 +- 已命中的技能不会出现在学习路径中。 + +### 确定性、空场景、身份与隐私 + +- `:123-133`：三个端点重复调用后 `data` 完全相等，并断言 8 张相关表的行数不变。 +- `:179-225`：事务内插入空分类，验证词云、匹配、学习路径分别返回空列表和 0 值。 +- `:227-240`：学生 JWT 额外传入 COMPANY 的 `userId`、`studentId`，三个端点结果均不被覆盖。 +- `:242-267`：成功响应不包含 `skills_json`、手机号、邮箱、简历 URL、简历原始文件名和 JWT；错误响应不包含异常类、Spring 内部路径、SQL 或堆栈。 + +### T9 findings + +- **Critical**：无。 +- **Important**：无。 +- **Minor T9-M1**：无写入断言以 8 张表行数前后不变为依据，可发现 INSERT/DELETE，但不能发现仅 UPDATE 同一行而不改变行数的情况。当前三个 Service 均为只读事务且测试通过，因此不阻断；后续可增加事务/数据库快照或只读约束断言。 +- **Minor T9-M2**：隐私测试未单独断言 `realName`、`bio` 的哨兵值，也未显式检查 `skillsJson` 驼峰名；已检查字段覆盖敏感号码、邮箱、简历标识、原始 `skills_json` 和 Token，风险为受控 DTO 下的低风险，不阻断。 +- **Minor T9-M3**：T9 的匹配一致性验证以结果中的 matchedIds 反推期望，因此主要证明“匹配结果与 TOP10 一致”，不独立证明具体学生技能命中哪一项标准技能；别名和匹配判定本身已由 T6 测试覆盖，不阻断。 + +### 测试隔离与假绿风险 + +- T9 类级 `@Transactional`，创建的 Student、StudentProfile 和测试用空分类在方法结束后回滚。 +- T9 使用独立 H2 URL，不与其他测试类共享数据库名。 +- 空分类使用固定测试 ID 并依赖事务回滚，没有观察到跨方法或跨类污染。 +- `@BeforeEach` 仅重置 Redis Mock，不依赖测试执行顺序。 +- 未发现共享 H2、事务传播或顺序依赖导致的假绿。 + +## 独立测试结果 + +### T2–T9 指定测试组 + +执行命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest,SkillCloudControllerTest,SkillCloudAuthorizationIntegrationTest,SkillCloudEndToEndIntegrationTest' test +``` + +独立结果： + +```text +Tests run: 71, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 01:08 min +``` + +分类统计： + +- SkillTaxonomyMigrationTest：2 +- SkillCloudSeedDataIntegrationTest：8 +- SkillTaxonomyRepositoryTest：6 +- SkillCloudQueryServiceTest：8 +- SkillMatchServiceTest：16 +- SkillLearningPathServiceTest：10 +- SkillCloudControllerTest：14 +- SkillCloudAuthorizationIntegrationTest：2 +- SkillCloudEndToEndIntegrationTest：5 + +合计：71。 + +### 全量测试 + +执行命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' test +``` + +独立结果： + +```text +Tests run: 1393, Failures: 2, Errors: 9, Skipped: 0 +BUILD FAILURE +Total time: 01:57 min +``` + +失败集合与实现者日志一致。 + +## 全量失败归因 + +### T9 阻断 + +无。全量失败集合中没有任何 Skill Cloud 测试或依赖 T9 测试数据的用例。 + +### 项目既有失败 + +#### 1. CareerProgressServiceTest.fullAggregation_returnsCorrectSummaries + +- 位置：`CareerProgressServiceTest.java:213`。 +- 现象：期望 pendingOfferResponseCount=1，实际=0。 +- 根因：测试构造的 Offer 固定过期时间 `2026-09-05T23:59:59`；当前日期为 `2026-09-15`，生产 `CareerProgressService.buildTodo()` 使用 `expireAt.isAfter(now)` 过滤，因此该 Offer 已过期。属于时间依赖的既有测试问题。 +- 与 N03 关系：该测试和生产类自 `bf75fc0` 后未在 `d00a630..HEAD` 中修改，且在不运行任何 Skill Cloud 测试时独立复现。 + +#### 2. JobReviewDimensionControllerTest.listReviews_shouldReturnDimensionScores + +- 位置：`JobReviewDimensionControllerTest.java:160-162`。 +- 现象：HTTP 200 期望，实际 500；`JobReviewController.list` 收到 null page。 +- 根因：Controller 已调用 7 参数 `listReviews(jobId, ratingType, createdFrom, createdTo, unreplied, page, size)`，测试仍 stub 旧的 3 参数 `listReviews(JOB_ID, 0, 20)`，Mockito 未命中导致 null。 +- 与 N03 关系：相关 Controller、Service 和测试均未在 `d00a630..HEAD` 中修改，且独立复现。 + +#### 3. JobReviewDimensionServiceTest.listReviews_shouldReturnDimensionScoresWithoutN1 + +- 位置：`JobReviewDimensionServiceTest.java:206-210`。 +- 现象：`reviewRepository.findPublicReviews(...)` 返回 null，触发 `JobReviewService.java:169` NPE。 +- 根因：测试仍 stub 旧方法 `findByJobIdAndStatusAndModerationStatusInOrderByCreatedAtDesc(...)`，与当前实现调用的 `findPublicReviews(...)` 不一致。 +- 与 N03 关系：相关 Repository、Service 和测试均未受 N03 变更修改，且独立复现。 + +### 环境阻塞 + +#### 4. LocalResumeStorageServiceTest + +- 现象：8 项测试在 JUnit extension context 清理阶段报 `AccessDeniedException`，路径为 Windows 临时目录下的 `junit-*`。 +- 根因：`WindowsPath.toRealPath()` 在测试 `@TempDir` 清理时访问被拒，测试业务逻辑本身未报断言失败。 +- 与 N03 关系：属于 Windows 临时目录权限/沙箱环境问题；相关类和测试自初始提交后未受 N03 修改，且独立复现。 + +## 门禁决定 + +- **T9**：PASS。T9 测试具备真实持久化链路、TOP10 一致性、学习路径顺序、确定性、空场景、身份和隐私断言，并已通过 T2–T9 指定测试组。 +- **提交 T9**：允许仅提交 `SkillCloudEndToEndIntegrationTest.java` 及本评审报告；不得把当前工作区其他无关改动混入该提交。 +- **继续 T10**：允许进入 T10 的实现与文档工作，但必须先明确处理全量 `mvn test` 的 11 个非 T9 失败。可以将其登记为项目既有/环境阻断并单独修复，或在项目层面正式接受并记录豁免；在此之前不得宣称 TO-11、B3 或发布级验证通过，也不得执行最终收尾。 +- **剩余阻断**：仅存在项目级全量回归阻断，不存在 T9 阻断。 + +## 证据限制 + +- 本次验证使用 H2 `MODE=MySQL` 和 Flyway，未执行真实 MySQL 黑盒迁移；MySQL 验证仍应在 T10/发布级验证门禁中完成。 +- 本次评审未修改任何实现或测试源码。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave-final-review.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave-final-review.md new file mode 100644 index 0000000..7b7c20f --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave-final-review.md @@ -0,0 +1,168 @@ +# N03 SDD W3 最终独立复审报告 + +## 复审对象 + +- 变更：`n03-skill-cloud` +- 基线：`df7238b` +- 复审焦点：`W3-N-I1` 修复及其对 `W3-I1`、`W3-I2` 的回归影响。 +- 工作区对象：`docs/openapi-rc.json`、`docs/verify_openapi.py`，并核对 `SkillCloudController` 和权限测试中的既有修复。 +- 环境：JDK 17（`D:\java\jdk`）、Maven 3.9.7（`D:\maven\apache-maven-3.9.7\bin\mvn.cmd`）。 +- 前轮报告：`changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave-rereview.md`。 + +## Verdict + +**PASS** + +`W3-N-I1` 已关闭。两个既有 `cancel` operationId 已恢复，静态 OpenAPI 不再受无关历史契约变更影响；Skill Cloud 三个 operationId 的冲突检查仍有效。`W3-I1`、`W3-I2` 保持关闭。W3 代码可以进入收尾和提交阶段。 + +该 PASS 仅针对 W3/N-I1 代码与契约修复，不代表完整项目回归通过。完整 `mvn test` 仍有 2 failures + 9 errors，变更最终关闭或 TO-11 前仍需修复或正式豁免。 + +## N-I1 关闭证据 + +### 既有 cancel 契约已恢复 + +当前 `docs/openapi-rc.json`： + +- `POST /interviews/{interviewId}/cancel`：`operationId = cancel` +- `POST /profile/me/deletion/cancel`：`operationId = cancel` + +对三个 Git 状态进行了全量 operationId 对比： + +```text +df7238b -> HEAD +新增： +/skills/word-cloud getWordCloud +/skills/match getMatch +/skills/learning-path getLearningPath + +HEAD -> 当前工作区 +仅： +/skills/match getMatch -> getSkillMatch + +df7238b -> 当前工作区 +仅三个新增 Skill Cloud operationId，其中 /skills/match 为 getSkillMatch +``` + +因此： + +- 两个历史 `cancel` 值已恢复，没有再次被改名为 `cancelInterview` 或 `cancelAccountDeletion`。 +- 除三个 Skill Cloud operationId 外，没有其他 operationId 在基线 `df7238b` 到当前工作区之间发生变化。 +- `docs/verify_openapi.py` 已移除全文档唯一性要求，只对三个 Skill Cloud operationId 逐个执行全文档“恰好出现于对应路径”的断言。 +- 历史重复值 `cancel` 不再阻断校验，但 Skill Cloud 的新 operationId 仍不能与其他 operation 冲突。 + +结论：**N-I1 已关闭。** + +## W3-I1 保持关闭 + +当前契约与运行时测试保持一致： + +- `/skills/word-cloud` -> `getWordCloud` +- `/skills/match` -> `getSkillMatch` +- `/skills/learning-path` -> `getLearningPath` + +`SkillCloudControllerTest` 在运行时 `/v3/api-docs` 上校验三个 operationId 的精确值及其唯一归属；`docs/verify_openapi.py` 对静态 OpenAPI 执行同等约束。前轮的 `getMatch` 重复冲突没有回退。 + +结论：**W3-I1 保持关闭。** + +## W3-I2 保持关闭 + +401 契约未回退： + +- 运行时匿名请求和非法 JWT 均返回 `code=401`、`message=not authenticated`、`timestamp=""`。 +- 响应不包含 `requestId` 或 `data`。 +- `SkillCloudAuthorizationIntegrationTest` 同时覆盖匿名和 `Bearer invalid-token`。 +- 静态 OpenAPI 中三个 Skill Cloud 接口的 401 均引用 `UnauthorizedResponse`，示例与运行时一致。 +- `docs/verify_openapi.py` 继续校验 401 schema、精确响应字段和示例内容。 + +结论：**W3-I2 保持关闭。** + +## 独立验证结果 + +### 静态 OpenAPI 校验 + +命令： + +```powershell +python docs\verify_openapi.py +``` + +结果： + +```text +paths: 129 +All checks passed! +``` + +通过。 + +### T2-T9 指定测试组 + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest,SkillCloudControllerTest,SkillCloudAuthorizationIntegrationTest,SkillCloudEndToEndIntegrationTest' test +``` + +结果： + +```text +Tests run: 73, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 54.628 s +``` + +通过。 + +### 打包 + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' -DskipTests package +``` + +结果： + +```text +BUILD SUCCESS +Total time: 2.387 s +``` + +通过。 + +## 项目级 blocker + +完整回归命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' test +``` + +结果： + +```text +Tests run: 1395, Failures: 2, Errors: 9, Skipped: 0 +BUILD FAILURE +Total time: 01:50 min +``` + +失败项： + +- `CareerProgressServiceTest.fullAggregation_returnsCorrectSummaries`：期望 1，实际 0，时间依赖问题。 +- `JobReviewDimensionControllerTest.listReviews_shouldReturnDimensionScores`：期望 200，实际 500，旧 Mock 签名问题。 +- `JobReviewDimensionServiceTest.listReviews_shouldReturnDimensionScoresWithoutN1`：返回结果 null 导致 NPE，旧 Repository stub 问题。 +- `LocalResumeStorageServiceTest`：8 项错误，Windows JUnit 临时目录清理阶段异常。 + +Skill Cloud 目标测试在完整套件中均通过，但完整 `mvn test` 未全绿。上述 2 failures + 9 errors 仍是项目级 blocker；在修复或正式豁免前，不得声称完整回归通过，也不得进入最终变更关闭或 TO-11。 + +## 收尾结论 + +- W3-N-I1：**关闭** +- W3-I1：**保持关闭** +- W3-I2：**保持关闭** +- 是否允许 W3 代码收尾：**允许** +- 是否允许最终变更关闭：**不允许**，完整 `mvn test` 仍有 2 failures + 9 errors，需修复或正式豁免。 +- 提交范围要求：仓库中存在与本复审无关的暂存、修改和未跟踪文件；W3 收尾提交时必须只包含 N03 范围内的文件，避免混入无关变更。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave-rereview.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave-rereview.md new file mode 100644 index 0000000..9f72513 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave-rereview.md @@ -0,0 +1,178 @@ +# N03 SDD W3 聚焦复审报告 + +## 复审对象 + +- 代码基线：`cd9220b`，修复差异仍位于工作区，未提交。 +- 复审范围：W3-I1、W3-I2 两项 Important finding 的聚焦修复。 +- 环境：`JAVA_HOME=D:\java\jdk`，Maven `D:\maven\apache-maven-3.9.7\bin\mvn.cmd`。 +- 对照报告：`changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave.md`。 + +## Verdict + +**FAIL** + +W3-I1 的原始 `/skills/match` operationId 冲突已修复，W3-I2 要求的运行时 401 契约、Markdown 文档和静态 OpenAPI 已对齐；但本次为满足全局 operationId 唯一性而修改了两处既有 `cancel` 契约，新增一项 Important 问题。 + +因此 W3 仍不能判定通过，不能进入收尾或 TO-11。 + +## W3-I1：operationId 冲突 + +### 关闭状态 + +**原始 finding 已关闭。** + +- `SkillCloudController.java:44` 显式设置 `operationId=getWordCloud`。 +- `SkillCloudController.java:92` 显式设置 `operationId=getSkillMatch`。 +- `SkillCloudController.java:140` 显式设置 `operationId=getLearningPath`。 +- 静态 OpenAPI 三个 Skill Cloud operationId 与上述值完全一致： + - `/skills/word-cloud` = `getWordCloud` + - `/skills/match` = `getSkillMatch` + - `/skills/learning-path` = `getLearningPath` +- `docs/verify_openapi.py:165-168` 对三个 operationId 做精确断言。 +- `SkillCloudControllerTest.skillOperationIdsAreExactAndGloballyUnique` 通过运行时 `/v3/api-docs` 校验三个接口的 operationId，并确认每个目标 operationId 只对应目标路径。 + +### 新增问题 N-I1（Important）：修改了既有 OpenAPI 契约 + +`docs/verify_openapi.py:27-31` 新增了对全文档 operationId 的无条件唯一性断言。为让该断言通过，本次修复还修改了与 N03 无关的两个既有接口： + +- `POST /interviews/{interviewId}/cancel`：`cancel` → `cancelInterview` +- `POST /profile/me/deletion/cancel`：`cancel` → `cancelAccountDeletion` + +问题如下： + +1. `operationId` 属于 OpenAPI 公共契约，常用于客户端和 SDK 生成。修改既有 operationId 属于兼容性破坏，即使 HTTP 路径和运行时行为不变，也可能导致生成客户端的方法名变化或升级失败。 +2. 两个 `cancel` 重复值是本次变更前已经存在的历史问题，不是 N03 引入的问题，也不在 W3-I1 的修复范围内。 +3. `InterviewController.java:132-136` 和 `AccountDeletionController.java:112-115` 仍使用方法名 `cancel`，且没有显式 operationId；因此静态 OpenAPI 被改为两个新名称后，静态契约与运行时 SpringDoc 文档发生漂移。 +4. `SkillCloudControllerTest.skillOperationIdsAreExactAndGloballyUnique` 的名称虽然包含 “GloballyUnique”，但实际只验证三个 Skill Cloud operationId 各自只出现一次，并没有验证全文档唯一性。真正强制全文档唯一性的是 `verify_openapi.py`，而它迫使本次变更修改历史契约。 + +**严重级别：Important。** + +**最小修复建议：** + +- 将 `docs/openapi-rc.json` 中两个既有接口的 operationId 恢复为 `cancel`，不要在本变更中修改既有接口契约。 +- 将 `docs/verify_openapi.py` 的检查改为“新增接口不得产生新冲突”，而不是“修复全部历史冲突”。例如： + - 保留已知历史重复集合 `{'cancel'}`； + - 断言除该允许集合外没有其他重复 operationId； + - 断言 `getWordCloud/getSkillMatch/getLearningPath` 各出现且仅出现一次； + - 断言它们不与历史或未来任何非允许 operationId 冲突。 +- 如果项目要求彻底消除全局 operationId 重复，应另开变更，同时在两个既有 Controller 上显式设置 operationId，并评估客户端兼容性与迁移方案。 + +## W3-I2：401 运行时与文档契约 + +### 关闭状态 + +**本次要求范围内的 finding 已关闭。** + +- 运行时实现仍由全局 Security 入口返回： + - `code=401` + - `message="not authenticated"` + - `timestamp=""` + - 不包含 `requestId` + - 不包含 `data` +- `SkillCloudControllerTest.anonymousRequestUsesSecurityEntryPointContract` 对三个接口执行上述字段级断言，并额外断言响应字段集合恰好为 `code/message/timestamp`。 +- `SkillCloudAuthorizationIntegrationTest.anonymousAndInvalidJwtUseSecurityEntryPointContract` 同时覆盖匿名请求和 `Bearer invalid-token`。 +- `docs/skill-cloud-api.md` 已明确区分 `401` 裸 JSON 与统一 `ApiResponse`，示例与运行时一致。 +- 静态 OpenAPI 三个 Skill Cloud `401` 响应均引用 `UnauthorizedResponse`，示例均为： + - `code=401` + - `message="not authenticated"` + - `timestamp=""` + - 无 `requestId`、无 `data` +- `docs/verify_openapi.py:173-185` 精确校验上述静态 401 契约。 + +### 非阻断观察（Minor） + +`SkillCloudController.java:66、114、162` 的 401 注解仍使用 `@Schema(implementation = ApiResponse.class)`。因此运行时 `/v3/api-docs` 的 401 schema 仍可能展示通用 `ApiResponse`，而静态 OpenAPI 已改为 `UnauthorizedResponse`。本次要求明确检查的是 Markdown 文档和静态 OpenAPI，且该问题不阻碍运行时行为或本 finding 的既定修复；如运行时 API 文档也被视为发布契约，应另行聚焦修正控制器注解并补充运行时 OpenAPI 断言。 + +## 独立验证结果 + +### T2–T9 目标测试 + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest,SkillCloudControllerTest,SkillCloudAuthorizationIntegrationTest,SkillCloudEndToEndIntegrationTest' test +``` + +结果： + +```text +Tests run: 73, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 55.399 s +``` + +分类统计： + +- SkillTaxonomyMigrationTest：2 +- SkillCloudSeedDataIntegrationTest：8 +- SkillTaxonomyRepositoryTest：6 +- SkillCloudQueryServiceTest：8 +- SkillMatchServiceTest：16 +- SkillLearningPathServiceTest：10 +- SkillCloudControllerTest：15 +- SkillCloudAuthorizationIntegrationTest：3 +- SkillCloudEndToEndIntegrationTest：5 + +### OpenAPI 静态校验 + +命令： + +```powershell +python docs\verify_openapi.py +``` + +结果： + +```text +paths: 129 +All checks passed! +``` + +结论：静态校验通过，但该通过依赖于 N-I1 所描述的既有 operationId 修改，不能据此判定 W3 通过。 + +### 打包 + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' -DskipTests package +``` + +结果： + +```text +BUILD SUCCESS +``` + +### 完整 `mvn test` 项目级回归 + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' test +``` + +结果： + +```text +Tests run: 1395, Failures: 2, Errors: 9, Skipped: 0 +BUILD FAILURE +``` + +所有 Skill Cloud 目标测试在全量套件中通过；失败仍集中在既有或环境相关测试： + +- `CareerProgressServiceTest.fullAggregation_returnsCorrectSummaries`：时间依赖失败。 +- `JobReviewDimensionControllerTest.listReviews_shouldReturnDimensionScores`：旧 Mock 签名导致 500。 +- `JobReviewDimensionServiceTest.listReviews_shouldReturnDimensionScoresWithoutN1`：旧 Repository stub 导致 NPE。 +- `LocalResumeStorageServiceTest`：Windows JUnit 临时目录 `AccessDeniedException`，8 个错误。 + +因此不能声称完整 `mvn test` 通过。上述 2 failures + 9 errors 仍是项目级 blocker，未形成正式豁免前不能关闭变更或进入 TO-11。 + +## 剩余阻断项 + +1. **N-I1（Important）**：恢复既有 `cancel` operationId，并调整校验脚本，使唯一性检查只拦截新增冲突，不以修改历史契约为代价。 +2. **项目级验证 blocker**：完整 `mvn test` 仍有 2 failures + 9 errors，需修复或正式豁免。 +3. **Minor**：运行时 `/v3/api-docs` 的 401 schema 仍可能需要从通用 `ApiResponse` 切换为 `UnauthorizedResponse`，以避免静态/运行时 OpenAPI 漂移。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave.md new file mode 100644 index 0000000..f00cc0d --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/W3-wave.md @@ -0,0 +1,198 @@ +# N03 SDD W3 整批独立评审报告 + +## 评审对象 + +- 范围：`df7238b..cd9220b` +- Wave：W3 +- 批次：B3 / T8、T9、T10 +- 提交： + - `3e353bd feat: expose student skill cloud endpoints` + - `7873a42 test: add skill cloud end-to-end regression` + - `cd9220b docs: publish skill cloud api contract` +- 评审依据：修订后的 `execution-contract.md`、`design.md`、`tasks.md`、三份 spec，以及 `W3-T8*`、`W3-T9.md`。 +- 环境：`JAVA_HOME=D:\java\jdk`，Maven `D:\maven\apache-maven-3.9.7\bin\mvn.cmd`。 + +## Verdict + +**FAIL** + +W3 的运行时接口、STUDENT 权限、参数校验、DTO 序列化、端到端一致性和打包均通过独立验证；但 T10 发布的 OpenAPI 契约存在两项 Important 问题： + +1. 新增 `/skills/match` 的 `operationId=getMatch` 与既有 `/applications/{id}/match` 冲突。 +2. 匿名或无效 JWT 的运行时 401 响应为 `code=401` 的裸 JSON，而文档和静态 OpenAPI 声明 `2001` 及统一 `ApiResponse` 语义。 + +因此 T10 不能判定通过，且本 wave 不允许进入收尾或 TO-11。 + +## T8：Controller、权限、参数与 ApiResponse + +### 通过项 + +- 三个路由存在且均为 GET： + - `/skills/word-cloud` + - `/skills/match` + - `/skills/learning-path` +- `industryId`、`occupationId` 为必填正整数；`categoryId` 为可选正整数。 +- 成功后统一返回 `ResponseEntity>`，并保留 `X-Request-Id`。 +- `SecurityConfig` 仅放行已认证请求到 `/skills/**`；Controller 再通过真实 `ProfileService` 按数据库角色校验 STUDENT。 +- `SkillCloudAuthorizationIntegrationTest` 使用真实 `UserRepository`、数据库角色集合、JWT 过滤器和 `ProfileService`，没有 mock 核心权限链路。 +- 已覆盖 STUDENT=200、COMPANY=403、ADMIN=403、STUDENT+COMPANY=200、COMPANY+ADMIN=403。 +- 额外传入 `userId`、`studentId` 不能覆盖 JWT 主体。 +- `SkillCloudControllerTest` 共 14 项，授权集成测试共 2 项，独立执行均通过。 +- 未发现 B3 修改已有接口语义或引入无关模块变更。提交范围仅涉及新增 Controller、SecurityConfig 一条 GET 规则、技能测试和文档/OpenAPI。 + +### 阻断项 + +#### W3-I1：`operationId=getMatch` 重复 + +- `docs/openapi-rc.json:1186`：既有 `/applications/{id}/match` GET。 +- `docs/openapi-rc.json:6460`：新增 `/skills/match` GET。 +- 两处均使用 `operationId: "getMatch"`。 +- `SkillCloudController.java:120-121` 的方法名也是 `getMatch`，运行时生成的 operationId 很可能同样冲突；现有测试没有检查全局 operationId 唯一性。 +- `docs/verify_openapi.py:146` 反而明确断言新增接口的 operationId 必须是 `getMatch`，把冲突值固化为通过条件。 +- OpenAPI 规范要求 operationId 全局唯一；重复值会导致客户端/文档代码生成器出现歧义或相互覆盖。 + +修复方向：给 `SkillCloudController.getMatch` 显式设置唯一 operationId，例如 `getSkillMatch`，同步静态 OpenAPI 和 verify 脚本，并增加全局 operationId 唯一性断言。 + +#### W3-I2：401 运行时响应与文档/OpenAPI 不一致 + +- 运行时匿名请求由 `SecurityConfig.java:50-54` 直接写出 `{"code":401,...}`，且不包含 `requestId`。 +- `SkillCloudControllerTest.java:158-164` 明确断言匿名 401 的 `$.code=401`。 +- `docs/skill-cloud-api.md:14` 声明使用统一 `ApiResponse`。 +- `docs/skill-cloud-api.md:230-240` 声明 401 的典型业务码为 `2001`，示例也返回 `code=2001` 和 `requestId`。 +- `docs/openapi-rc.json` 的三个新增 401 响应示例均使用 `code=2001`。 +- 因此静态契约与真实运行时对相同的“未登录/无效 JWT”场景给出了不同的业务码和响应语义。 + +修复方向：二选一并做可证伪测试： + +- 保持现有全局安全行为，更新接口文档和静态 OpenAPI 为真实 `code=401` 响应；或 +- 在授权范围内调整安全入口，使 401 使用统一的 `ApiResponse` 和 `2001`，同时评估对所有既有接口的兼容性。 + +## T9：端到端一致性 + +### 通过项 + +`SkillCloudEndToEndIntegrationTest` 共 5 项，使用真实 H2/Flyway、Security、Controller、Service、Repository 和 V36 数据，仅 Mock `RedisTemplate` 以隔离基础设施依赖，独立执行通过。 + +已验证： + +- 词云返回完整排序列表，`rank` 连续，`heat DESC`，空热度末位。 +- 匹配集合严格等于 `min(10,total)`，`matchedSkills + missingSkills` 等于同一 TOP10，且各自保持顺序。 +- 学习路径严格复用 `missingSkills` 顺序，`priority` 从 1 连续递增。 +- 少于 10 项时按实际大小计算，`1/3` 返回 33；空范围返回 0 和空列表。 +- 重复调用三个接口返回完全相同的数据。 +- 合法空分类覆盖 `total=0`、`matchScore=0`、`learningPath.total=0`。 +- 成功响应未发现手机号、邮箱、简历文件名/URL 或原始技能 JSON。 +- 额外身份查询参数不能覆盖学生 JWT 主体。 + +### 非阻断测试缺口 + +- “无写入”目前只比较 8 张表的行数，只能发现 INSERT/DELETE，不能发现仅 UPDATE 的写操作。 +- 隐私断言未显式覆盖 `realName`、`bio`、`skillsJson` 等驼峰或实际值。 +- 端到端测试主要证明三个接口复用同一 TOP10，具体名称/别名匹配分支由 T6 测试覆盖，这一点可以接受。 + +以上均为可加强项，不单独阻断 T9 运行时实现。 + +## T10：文档与 OpenAPI + +### 通过项 + +- 静态 OpenAPI path 数从 126 增至 129，三个新路径均存在。 +- 三个新路径均声明 `bearerAuth`、200/400/401/403，以及必填/可选参数。 +- 新增 Schema 字段与 Java DTO 字段在静态检查中一致。 +- `docs/skill-cloud-api.md` 和 `docs/frontend-integration.md` 覆盖 TOP10 截取、匹配公式、排序、空数据、权限、无 AI 和无持久化语义。 +- `python docs\verify_openapi.py` 独立执行成功，输出 `paths: 129` 和 `All checks passed`。 +- 未发现三个新接口之外的接口语义被改变。 + +### 阻断项 + +见上方 `W3-I1`、`W3-I2`。T10 当前不能判定通过。 + +### verify 脚本的不足 + +- `docs/verify_openapi.py:164` 使用 `expected_fields <= properties`，只验证字段至少存在，不验证字段集合完全相等、类型、nullable、required 或静态契约与运行时的对应关系；新增多余字段仍可通过。 +- 脚本没有全局 operationId 唯一性断言，因此 `W3-I1` 未被发现。 +- 脚本检查静态 OpenAPI，而 `SkillCloudControllerTest` 检查运行时 `/v3/api-docs`，两者没有做等价性比较；当前静态文件使用 `example`，运行时注解生成使用 `examples`，进一步说明两者可能各自漂移。 +- 精确 path 数断言可以用于回归基线，但不能替代 operationId 唯一性、Schema 完整性和错误响应一致性验证。 + +## 独立执行结果 + +### T2–T9 指定测试组 + +命令： + +```powershell +$env:JAVA_HOME='D:\java\jdk' +& 'D:\maven\apache-maven-3.9.7\bin\mvn.cmd' '-Dtest=SkillTaxonomyMigrationTest,SkillCloudSeedDataIntegrationTest,SkillTaxonomyRepositoryTest,SkillCloudQueryServiceTest,SkillMatchServiceTest,SkillLearningPathServiceTest,SkillCloudControllerTest,SkillCloudAuthorizationIntegrationTest,SkillCloudEndToEndIntegrationTest' test +``` + +结果： + +```text +Tests run: 71, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 59.133 s +``` + +分类： + +- SkillTaxonomyMigrationTest：2 +- SkillCloudSeedDataIntegrationTest：8 +- SkillTaxonomyRepositoryTest：6 +- SkillCloudQueryServiceTest：8 +- SkillMatchServiceTest：16 +- SkillLearningPathServiceTest：10 +- SkillCloudControllerTest：14 +- SkillCloudAuthorizationIntegrationTest：2 +- SkillCloudEndToEndIntegrationTest：5 + +### OpenAPI 校验 + +```text +python docs\verify_openapi.py +paths: 129 +All checks passed! +``` + +该命令通过，但不覆盖上述 operationId 冲突、401 运行时差异和静态/运行时 Schema 等价性，因此不能抵消 W3-I1、W3-I2。 + +### 打包 + +```text +mvn -DskipTests package +BUILD SUCCESS +Total time: 2.401 s +``` + +### 完整 `mvn test` + +独立运行结果： + +```text +Tests run: 1393, Failures: 2, Errors: 9, Skipped: 0 +BUILD FAILURE +Total time: 01:58 min +``` + +Skill Cloud 全部测试类在该套件中均通过。 + +## 全量 blocker 归类 + +以下失败均不在 `df7238b..cd9220b` 修改范围内，不能归因于 W3 引入的行为回归： + +- `CareerProgressServiceTest.fullAggregation_returnsCorrectSummaries`：固定 Offer 过期时间 `2026-09-05` 已早于当前日期，属于时间依赖测试问题。 +- `JobReviewDimensionControllerTest.listReviews_shouldReturnDimensionScores`：仍 stub 旧服务方法签名。 +- `JobReviewDimensionServiceTest.listReviews_shouldReturnDimensionScoresWithoutN1`：仍 stub 旧 Repository 方法，导致 NPE。 +- `LocalResumeStorageServiceTest`：8 项均在 Windows JUnit 临时目录清理阶段因 `AccessDeniedException` 报错，属于环境/临时目录问题。 + +归类结论： + +- **不是 W3 代码阻断。** +- **仍然是项目级验证阻断，应阻止 `closing`/TO-11**，除非先修复，或由项目层面正式记录并批准豁免；合同退出条件不能被视为满足。 +- 本次独立验证未运行真实 MySQL 黑盒迁移，仍应保留为发布级验证风险。 + +## 最终放行结论 + +- **是否允许 T8 运行时实现进入后续：允许。** 权限、参数和运行时响应链路通过。 +- **是否允许 T9 端到端实现进入后续：允许。** 端到端行为通过，仅有非阻断覆盖缺口。 +- **是否允许 W3 整批通过：不允许。** T10 的 OpenAPI/文档契约存在 Important 阻断。 +- **是否允许进入收尾/TO-11：不允许。** 需要先修复 W3-I1、W3-I2 并进行聚焦复审；全量 `mvn test` 还需全绿或形成正式豁免记录。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/reviews/runtime-startup-fix-review.md b/changes/n03-skill-cloud/.superpowers/sdd/reviews/runtime-startup-fix-review.md new file mode 100644 index 0000000..46b17fa --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/reviews/runtime-startup-fix-review.md @@ -0,0 +1,151 @@ +# N03 运行时启动缺陷修复独立复审报告 + +## Verdict + +**PASS** + +运行时启动缺陷的本轮修复通过独立复审：实体不再强制 `TINYINT` 映射，V37 将 H2 侧列规范化为 `BOOLEAN`，MySQL 侧保持 `BOOLEAN/TINYINT(1)` 语义；标准 T2–T10 回归和跳过测试的打包均通过。完整 `mvn test` 仍有既有、非本次修复引入的失败，不能据此声称全量回归通过。 + +## 审查对象 + +- `mydreamjob-server/src/main/resources/db/migration/V37__skill_cloud_boolean_type.sql` +- `mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/OccupationSkill.java` +- `SkillTaxonomyRepositoryTest.java` +- `SkillTaxonomyMigrationTest.java` +- `SkillCloudSeedDataIntegrationTest.java` +- `SkillCloudQueryServiceTest.java` +- 已有证据日志：`target/skill-cloud-v37-red.log`、`skill-cloud-v37-final-green.log`、`skill-cloud-v37-final-package.log` + +## 根因闭环 + +原始启动失败为： + +```text +Schema-validation: wrong column type encountered in column [importance_is_proxy] +found [bit (Types#BIT)], but expecting [tinyint (Types#TINYINT)] +``` + +闭环判断： + +1. `OccupationSkill.importanceIsProxy` 原先使用显式 `@JdbcTypeCode(SqlTypes.TINYINT)`，把 Hibernate 的 JDBC 期望固定为 `TINYINT`。 +2. 本轮已移除该显式注解及其 import；字段仍是 `Boolean`，列名、可空性和其他字段映射未改变。 +3. 新回归测试同时断言： + - Hibernate 属性返回类型为 `Boolean.class`； + - 字段不再存在显式 `JdbcTypeCode` 注解。 +4. V37 将 `occupation_skills.importance_is_proxy` 规范化为 `BOOLEAN NOT NULL DEFAULT FALSE`，并重建原 CHECK 约束。H2 `MODE=MySQL` 下迁移测试确认元数据为 `BOOLEAN / FALSE / NO`。 +5. MySQL 8.0.43 临时实例实际执行同一 DDL 后，物理元数据为 `tinyint / tinyint(1) / default 0 / NO`，读写结果为默认 `0`、`FALSE=0`、`TRUE=1`。 + +因此根因修复成立：实体不再强制 TINYINT；H2 使用真正的 BOOLEAN；MySQL 保留 BOOLEAN 别名对应的 TINYINT(1) 物理语义。 + +## V35/V36 与 Flyway 安全检查 + +独立比较结果： + +- `V35__skill_cloud_taxonomy.sql`：工作区与 `HEAD` 的 Git hash 均为 `db81f934...`，无差异。 +- `V36__skill_cloud_seed.sql`：工作区与 `HEAD` 的 Git hash 均为 `e4a03a40...`，无差异。 +- V35/V36 均未改写，已应用环境的 Flyway checksum 不受影响。 +- V37 为追加迁移，未修改历史迁移内容。 + +H2 `MODE=MySQL` 兼容性由实际迁移与测试上下文启动验证；MySQL 兼容性由 MySQL 8.0.43 实例实际执行 `DROP CONSTRAINT`、`MODIFY COLUMN BOOLEAN`、`ADD CONSTRAINT` 验证。MySQL 8.4 未在本机安装，未做 8.4 二进制验证；所用 DDL 语法属于 MySQL 8.0/8.4 的通用受支持语法。 + +## 布尔语义、默认值与 NULL 检查 + +V37 的顺序为： + +1. 删除旧的 `ck_occupation_skills_importance_proxy`； +2. 将列改为 `BOOLEAN NOT NULL DEFAULT FALSE`； +3. 重建 `CHECK (importance_is_proxy IN (FALSE, TRUE))`。 + +经验证： + +- 默认值：新行未显式赋值时为 `FALSE/0`。 +- 非空：迁移测试断言 `IS_NULLABLE=NO`，并将 NULL 更新验证为失败。 +- 布尔读写：`FALSE`、`TRUE` 可读写，查询结果与布尔比较均正确。 +- JDBC 断言：查询测试和种子数据测试已兼容 H2 返回 `Boolean` 以及旧驱动可能返回数值两种形式，不会因方言差异误判。 +- `preferred_boolean_jdbc_type=TINYINT`：源代码与配置中未发现该属性；标准测试命令不需要额外 `-D` 参数。 + +## 独立测试与打包结果 + +环境： + +```text +JAVA_HOME=D:\java\jdk +Maven=D:\maven\apache-maven-3.9.7\bin\mvn.cmd +``` + +### T2–T10 指定测试组 + +在 `mydreamjob-server` 下执行，未附加任何布尔类型属性： + +```text +mvn -Dtest=SkillCloudAuthorizationIntegrationTest,SkillCloudControllerTest,SkillCloudEndToEndIntegrationTest,SkillCloudQueryServiceTest,SkillCloudSeedDataIntegrationTest,SkillLearningPathServiceTest,SkillMatchServiceTest,SkillTaxonomyMigrationTest,SkillTaxonomyRepositoryTest test +``` + +结果： + +```text +Tests run: 74, Failures: 0, Errors: 0, Skipped: 0 +BUILD SUCCESS +Total time: 55.561 s +``` + +### 打包 + +```text +mvn -DskipTests package +BUILD SUCCESS +Total time: 2.439 s +``` + +### MySQL 8.0.43 冒烟 + +实际执行结果： + +```text +id importance_is_proxy +1 0 +2 0 +3 1 + +DATA_TYPE COLUMN_TYPE COLUMN_DEFAULT IS_NULLABLE +tinyint tinyint(1) 0 NO +``` + +说明：MySQL 将 `BOOLEAN` 规范化为 `tinyint(1)` 属正常行为；这与 V37 目标及 H2 的 `BOOLEAN` 元数据要求一致。 + +## 完整 `mvn test` 状态 + +独立执行完整命令： + +```text +mvn test +Tests run: 1396, Failures: 2, Errors: 9, Skipped: 0 +BUILD FAILURE +``` + +失败项仍为与本功能无关的既有问题： + +- `CareerProgressServiceTest.fullAggregation_returnsCorrectSummaries`：固定过期时间导致期望 1、实际 0。 +- `JobReviewDimensionControllerTest.listReviews_shouldReturnDimensionScores`：测试仍使用旧服务方法签名，返回 500。 +- `JobReviewDimensionServiceTest.listReviews_shouldReturnDimensionScoresWithoutN1`：测试仍 stub 旧 Repository 方法，结果为 null。 +- `LocalResumeStorageServiceTest`：Windows JUnit 临时目录清理的 8 个 `AccessDeniedException` 错误。 + +Skill Cloud 相关测试在完整套件中通过；但按退出条件，完整 `mvn test` 仍未全绿，不能声称全量回归通过。 + +## 范围与遮蔽检查 + +- 本次修复的生产代码变更仅涉及 `OccupationSkill` 和新增 V37；其余变更为相应专项测试的布尔兼容与回归断言。 +- 未修改 `SecurityConfig`、接口、DTO、业务查询逻辑或其他 schema 映射。 +- `SkillTaxonomyRepositoryTest` 原有的 schema filter 仍只限制 H2 测试环境的旧表验证范围，本轮没有扩大或借助该 filter 掩盖 V37 相关 schema 问题；V37 专项迁移测试直接检查 `information_schema` 元数据和约束。 +- 工作区中 `.trae/*` 等文件属于本任务之外的既有未提交内容，不构成本次运行时修复差异。 + +## 非阻断观察 + +1. `V37__skill_cloud_boolean_type.sql` 当前索引版本与工作区版本存在一个仅涉及列注释的文本差异；两者不影响 DDL 语义，但提交前应统一内容，避免最终提交与独立测试的具体文本不一致。 +2. V37 文件工作区版本末尾缺少换行；非功能问题。 +3. MySQL 仅完成 8.0.43 二进制冒烟，未在实际 8.4 二进制上复跑；判断为语法兼容，但如需严格的 8.4 认证应补充运行环境验证。 + +## 剩余阻断 + +- 本次运行时启动修复：无阻断。 +- 项目最终关闭：完整 `mvn test` 仍有 2 failures + 9 errors；需要修复这些既有测试，或由项目按流程正式记录豁免后，才能推进全量回归结论和最终关闭。 diff --git a/changes/n03-skill-cloud/.superpowers/sdd/runtime-startup-failure.txt b/changes/n03-skill-cloud/.superpowers/sdd/runtime-startup-failure.txt new file mode 100644 index 0000000..321ef80 --- /dev/null +++ b/changes/n03-skill-cloud/.superpowers/sdd/runtime-startup-failure.txt @@ -0,0 +1,14 @@ +Runtime failure evidence +======================= +Task: manual application startup from IntelliJ +Symptom: Hibernate SessionFactory build failed +Column: occupation_skills.importance_is_proxy +Found: bit (Types#BIT) +Expected: tinyint (Types#TINYINT) +Root cause: OccupationSkill.importanceIsProxy is explicitly forced to SqlTypes.TINYINT, unlike existing Boolean mappings. + +Attempt 2 evidence +================== +Removing @JdbcTypeCode(TINYINT) alone fixed the MySQL expectation but broke H2 schema validation: +found tinyint, expected boolean. +Conclusion: the column must be normalized to a cross-dialect BOOLEAN/BIT form, without modifying already-applied V35. diff --git a/changes/n03-skill-cloud/data/source-manifest.md b/changes/n03-skill-cloud/data/source-manifest.md new file mode 100644 index 0000000..12b68d9 --- /dev/null +++ b/changes/n03-skill-cloud/data/source-manifest.md @@ -0,0 +1,77 @@ +# N03 行业技能词云来源与覆盖清单 + +## 1. 固定来源 + +| 来源 | 固定版本 | 用途 | 许可/复用条件 | 官方地址与归属 | +| --- | --- | --- | --- | --- | +| ESCO | ESCO dataset v1.2.1（Last update 10/12/2025，发布页记录 Release v5.7.0） | 提供行业/职位映射所需的职业—技能—技能分类主关系、稳定来源 ID 和层级结构 | 依据 Commission Decision 2011/833/EU，可免费下载、使用、复制和复用。文档和产品界面须保留官方署名语句：`This service uses the ESCO classification of the European Commission.` | 数据集下载页：https://esco.ec.europa.eu/en/use-esco/download
复用条件说明：https://esco.ec.europa.eu/en/about-esco/faq | +| O*NET | O*NET 31.0 Database | 为 ESCO 未收录的具体技术提供 `software_skills` 补充来源，并通过职业/工作活动 Importance 提供首版词云热度 `heat` | License 为 CC BY 4.0，归属与许可说明见 https://creativecommons.org/licenses/by/4.0/ | 官方数据库页：https://www.onetcenter.org/database.html | + +运行时不得访问 ESCO、O*NET 或任何在线词库服务。所有首版数据必须随 Flyway 迁移进入本地数据库，并保留来源版本与归属信息。 + +## 2. 首版覆盖边界 + +首版只覆盖互联网、金融、教育三类行业及下列 10 个已列代表性职位。该矩阵定义导入上限和最小覆盖范围，不要求在全量上游数据中导入未列出的职位。 + +| 行业 | 代表性职位 | 首批技能分类范围 | +| --- | --- | --- | +| 互联网 | 前端开发工程师 | 编程语言、Web 基础、前端框架、工程化工具、浏览器与调试、测试、版本控制与协作 | +| 互联网 | 后端开发工程师 | 编程语言、后端框架与平台、数据库、缓存与消息、云/DevOps、测试、工程化工具、安全 | +| 互联网 | 测试工程师 | 编程语言、测试设计、自动化测试框架、接口与数据库测试、性能测试、CI/CD 与工程化工具 | +| 互联网 | DevOps 工程师 | 编程语言与脚本、云平台、容器与编排、CI/CD、配置管理与 IaC、可观测性、安全、数据库 | +| 金融 | 金融数据分析师 | 编程语言、数据分析与可视化、数据库与数据仓库、统计与建模、金融计算框架/平台、金融领域知识、工程化工具 | +| 金融 | 风控建模工程师 | 编程语言、统计与建模、风险模型、建模框架/平台、数据处理与数据库、模型验证、部署与工程化工具 | +| 金融 | 量化开发工程师 | 编程语言、量化建模框架/平台、市场与金融数据、数据库、低延迟与高性能工程、测试、云/DevOps | +| 教育 | 教育产品经理 | 教育领域知识、产品分析方法、需求与流程、数据分析、教育平台与工具、协作与工程化工具 | +| 教育 | 教学设计师 | 教学设计、课程与学习平台、数字化内容工具、评估设计、学习数据分析、无障碍与规范 | +| 教育 | 教育数据分析师 | 编程语言、数据分析与可视化、数据库与 SQL、统计与建模、教育领域知识、报表与工程化工具 | +计数说明：任务给出的逐项职位清单为互联网 4 个、金融 3 个、教育 3 个，共 10 个。验收文字中的“11 个职位”与该枚举不一致；本清单严格覆盖逐项列出的 10 个职位，不自行新增未批准职位。 + +矩阵中的分类是每个职位首批应覆盖的能力域；T3 只导入这些分类下已确定稳定来源 ID、标准技能和中文标准名的技能，不在此文档中展开具体 `INSERT` 行。 + +## 3. 导入规则 + +1. **职责分工**：T2 只建立七张词库表、外键、唯一约束和查询索引；T3 只导入上述三类行业、10 个已列代表性职位及其关联技能分类、技能、别名和 O*NET Importance。 +2. **代表性子集与混合来源**：不导入 ESCO/O*NET 全量数据；ESCO 是职位、通用技能和分类主表；ESCO 无可靠节点的具体技术使用 O*NET `software_skills` 补充；不把平台自由文本岗位或行业直接当作标准词库节点。 +3. **来源 ID 不变**：ESCO/O*NET 来源 ID 原样保存在稳定来源字段中；项目不得重编号、覆盖或伪造上游 ID。 +4. **中文标准名必填**：每个入选技能必须有非空 `name_zh` 中文标准名；中文名称由项目维护，但不替代或修改上游来源 ID。 +5. **热度来源**：每条非空 Importance 必须保存 O*NET SOC、Element ID 和代理标记；技能无法映射 O*NET Importance 时必须保留，写入 `importance=null`；查询时映射为 `heat=null`，并排在有热度技能之后。 +6. **项目映射责任**：项目维护中文名称、别名、技能分类和行业/职位映射，不修改开源来源 ID。行业/职位映射、细粒度分类和跨源技能对应关系均作为项目适配层记录。 +7. **别名归一化**：生成 `normalized_alias` 时依次执行 Unicode NFKC、首尾空白移除、大小写折叠、连字符转空格、连续空白折叠为单空格、句点移除；`+` 和 `#` 必须保留，不能删除或替换。 +8. **来源可追溯**：来源清单写入 `skill_source_versions`，至少记录来源名称、固定版本、许可、归属 URL 和导入时间；O*NET 补充技术的 `source_code` 使用真实 `SOC|Workplace Example|Element ID` 组合，职位技能关系记录 O*NET SOC、Element ID、代理标记及关联的来源版本。 +9. **迁移不可改写**：T2/T3 计划使用 `V35__skill_cloud_taxonomy.sql` 和 `V36__skill_cloud_seed.sql`。执行前若编号已被占用，必须回退规划并刷新编号；后续词库更新只能新增 Flyway 迁移，不得改写 V1–V34 或任何已执行迁移。 +10. **运行时离线**：接口运行时不得调用外部词库、AI、课程平台或同步服务；排名只读取本地 Flyway 数据。 + +## 4. 来源版本、许可与归属 URL + +| 数据落点 | 来源名称 | 版本字段固定值 | 许可字段固定值 | 归属 URL | +| --- | --- | --- | --- | --- | +| 职业、技能、技能分类及关系 | ESCO | `1.2.1 (Release v5.7.0, Last update 10/12/2025)` | `Commission Decision 2011/833/EU` | https://esco.ec.europa.eu/en/use-esco/download | +| 职位技能 Importance | O*NET | `31.0 Database` | `CC BY 4.0` | https://www.onetcenter.org/database.html | + +ESCO 产品署名必须使用：`This service uses the ESCO classification of the European Commission.` + +## 5. 映射责任边界 + +| 事项 | 上游来源责任 | 项目责任 | +| --- | --- | --- | +| ESCO 职业、技能、分类层级和来源 ID | 固定且可追溯 | 原样保存，不修改 ID | +| O*NET Importance 数值 | 固定且可追溯 | 原样保存；无映射时写 `null`，不造权重 | +| 中文名称和别名 | 不保证完整中文覆盖 | 补齐中文标准名、别名及归一化值 | +| 细粒度技能分类 | 不保证符合本项目展示粒度 | 在职位下维护分类和展示名称 | +| 行业/职位映射 | 不保证覆盖三类项目行业和中文职位名 | 维护行业—职位—分类—技能映射 | +| 跨源技能与重要性关联 | 不提供项目所需的稳定本地关系 | 维护 ESCO 技能到 O*NET `software_skills` 的补充映射，以及每条 Importance 的 SOC/Element ID/代理标记 | + +## 6. 验收检查项 + +- AC-1：`changes/n03-skill-cloud/data/source-manifest.md` 与 `docs/skill-taxonomy-sources.md` 均存在且非空，来源版本、许可、归属 URL、覆盖矩阵和导入规则一致。 +- AC-2：文档明确覆盖互联网、金融、教育三类行业及全部 10 个已列代表性职位，并为每个职位给出首批技能分类范围。 +- AC-3：文档明确固定 ESCO dataset v1.2.1（Last update 10/12/2025，发布页记录 Release v5.7.0）和 O*NET 31.0 Database。 +- AC-4：文档明确 ESCO 依据 Commission Decision 2011/833/EU 可免费下载、使用、复制和复用，并原样保留指定署名语句。 +- AC-5：文档明确 O*NET License 为 CC BY 4.0，并记录官方数据库页与许可 URL。 +- AC-6：文档明确项目维护中文名称、别名、技能分类和行业/职位映射，不修改开源来源 ID。 +- AC-7：文档明确 T2 建结构、T3 导数据，只导入代表性职位和关联技能；技能中文标准名必填。 +- AC-8：文档明确 O*NET 补充技术、每条 Importance 的 SOC/Element ID/代理标记，以及无映射时保留技能并写入 `importance=null`、查询侧映射为 `heat=null`。 +- AC-9：文档明确别名归一化遵循 NFKC、首尾空白、大小写、连字符、连续空白、句点规则，且保留 `+`、`#`。 +- AC-10：文档明确后续词库更新只能新增 Flyway 迁移，不得改写 V1–V34 或已执行迁移。 +- AC-11：来源、覆盖、导入与责任边界中不得出现未决占位或延期确认语句。 diff --git a/changes/n03-skill-cloud/decision-point-audit.md b/changes/n03-skill-cloud/decision-point-audit.md new file mode 100644 index 0000000..ebc7f93 --- /dev/null +++ b/changes/n03-skill-cloud/decision-point-audit.md @@ -0,0 +1,55 @@ +# N03 Decision Point Audit + +审计时间：2026-09-15T08:40:05Z + +## Decision Point Status + +| DP | Status | Evidence | +| --- | --- | --- | +| DP-0 | confirmed | 需求、范围、非目标、语言与 Full workflow 已确认 | +| DP-1 | confirmed | 三接口需求口径已确认 | +| DP-2 | approved | proposal、三份 delta specs、design、tasks 已审阅 | +| DP-3 | approved | execution contract revision 2 已批准 | +| DP-4 | approved | SDD；W1/W2/W3 串行 wave，依赖关系已固化 | +| DP-5 | adjudicated | T3 来源与可证伪性修复获人工批准 | +| DP-6 | pass with formal waiver | 功能范围内验证通过；全量测试中的既有无关失败由用户正式豁免 | +| DP-7 | pending | 等待归档确认 | + +## Contract Coverage + +- W1：来源清单、V35 词库结构、V36 代表性种子数据、可追溯 O*NET provenance。 +- W2：Repository、词云排序、个人匹配、学习路径，50 项 wave 测试通过。 +- W3：三个 STUDENT 接口、真实权限测试、端到端一致性、接口文档和 OpenAPI。 +- 14 条 MUST/SHALL 行为均有实现与测试证据。 +- delta specs 已发布到根 baseline：specs/skill-cloud、specs/skill-match、specs/learning-path，共 14 条 Requirements，无冲突。 + +## Verification Evidence + +- Skill Cloud 指定测试组：74 tests，0 failures，0 errors，BUILD SUCCESS。 +- mvn -DskipTests package：BUILD SUCCESS。 +- python docs\verify_openapi.py：paths 129，All checks passed。 +- 启动缺陷修复：V37 将 importance_is_proxy 规范化为跨方言 BOOLEAN；H2 MODE=MySQL 与 MySQL 8.0.43 冒烟通过。 +- 完整 mvn test：1396 tests，2 failures，1 error，BUILD FAILURE。 +- 本次完整测试失败： + - CareerProgressServiceTest.fullAggregation_returnsCorrectSummaries：既有时间依赖。 + - JobReviewDimensionControllerTest.listReviews_shouldReturnDimensionScores：既有 Mock 签名漂移。 + - JobReviewDimensionServiceTest.listReviews_shouldReturnDimensionScoresWithoutN1：既有 Repository 方法签名漂移。 +- 这些失败位于本变更未修改的模块；Skill Cloud 测试在全量套件中全部通过。 + +## Formal Waiver + +用户于 2026-09-15 明确批准：将与本功能无关的既有失败作为正式豁免记录，并进入 release-archivist。当前 fresh run 只剩上述 3 条历史阻塞；更早一轮曾包含 LocalResumeStorageServiceTest 的 Windows 临时目录错误，后续环境运行已恢复。本豁免覆盖所有与本功能无关、且在变更前后可独立复现的既有失败。 + +## Scope and Coherence + +- 变更实现集中在 com.mydreamjob.skill、Flyway V35–V37、SecurityConfig、docs、OpenAPI 与对应测试。 +- 未修改 V1–V34 历史迁移或已有接口语义。 +- V37 是启动兼容修复，不改变 API 行为或业务公式。 +- W2/W3 独立评审均已通过；运行时启动修复也已独立复审 PASS。 +- 已知非阻断项：heatSource 设计表述与实际 heatSourceCode/heatElementId/heatProxy 字段命名不同；文档已按实际字段发布。 + +## Remaining Actions + +1. 记录 DP-7 归档确认。 +2. 将状态从 executing 转为 closing。 +3. ssf CLI 不可用；物理归档/合并需在获得用户确认后手工执行，且必须排除仓库中已有的无关未提交文件。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/design.md b/changes/n03-skill-cloud/design.md new file mode 100644 index 0000000..cff25a3 --- /dev/null +++ b/changes/n03-skill-cloud/design.md @@ -0,0 +1,180 @@ +# 行业技能词云与学习路径设计 + +## 事实与约束 + +- 仓库当前为 Spring Boot 3.5.16、Java 17、MySQL/Flyway/JPA，最新已存在迁移为 `V34`。 +- 学生技能存储在 `student_profiles.skills_json`，历史数据可能为空、重复或无法解析。 +- 企业行业与岗位业务数据是自由文本，不能作为稳定的“行业 → 职位 → 技能”分类源。 +- ESCO 提供职位、技能和技能分类关系；O*NET 提供 Importance 数值。两者均需保留来源版本与许可归属。 +- 首版只导入互联网、金融、教育及代表性职位，避免全量翻译和超大迁移。 +- 三个接口均为只读、仅 STUDENT，身份只取 JWT，响应统一使用 `ApiResponse`。 +- 词云热度是静态 O*NET Importance，不随平台岗位发布和投递行为变化。 +- 计算完全确定，不调用 AI、网络课程服务或第三方运行时接口。 + +## Goals + +- 建立可持续扩展的开源技能知识库，并允许项目维护中文名称、别名和细粒度分类。 +- 提供完整、稳定、可筛选的热度排序技能词云。 +- 以同一排序结果计算个人 TOP10 匹配度，避免不同接口结果不一致。 +- 基于未命中技能生成最小可用的学习优先级和建议。 +- 保持现有模块接口语义不变，并满足项目测试、OpenAPI 和黑盒验收约定。 + +## Non-Goals + +- 不做岗位实时文本挖掘、平台岗位热度聚合或用户行为周榜。 +- 不导入 ESCO/O*NET 全量数据，不建设后台维护界面。 +- 不保存匹配快照、学习进度、完成状态或提醒。 +- 不提供课程、讲师、价格、时长、链接和 AI 对话。 +- 不向 COMPANY/ADMIN 暴露个人匹配和学习路径。 + +## Decisions + +### D1: 以 ESCO + O*NET 作为静态知识底座 + +**Choice**：ESCO 作为职位、通用技能和分类主表；当具体技术（如 React、Docker、Spring Boot）没有可靠 ESCO 节点时，允许使用 O*NET `software_skills` 作为补充技能来源。O*NET 继续提供可追溯 Importance；项目只维护中文标签、别名和映射。 + +**Rationale**：能用开源数据覆盖技能层级，同时保留 O*NET 明确的重要度字段，不需要自行创造热度权重。 + +**Alternatives**： +- 仅 O*NET：重要度完整，但英文和美国职业体系适配成本高。 +- 严格仅 ESCO：来源单一，但会丢失 React、Docker 等 ESCO 未收录的具体技能。 +- 自建词典：可控性高，但需要长期运营且缺少来源可信度。 + +**Consequences**：需要明确区分 ESCO 主来源技能与 O*NET 补充技术；每条热度必须记录 O*NET SOC 和 Element ID，无法映射时保留技能并写空热度。 + +### D2: 词库使用关系表落 Flyway + +**Choice**：新增行业、职位、技能分类、技能、别名、职位技能关系和来源版本表，通过 Flyway 迁移导入代表性数据。 + +**Rationale**：接口需要多条件过滤、稳定排序和可追溯来源；关系表可利用外键、唯一约束和索引保证数据质量。 + +**Alternatives**： +- 静态 JSON：实现快，但无法方便地按节点过滤、校验引用和演进。 +- 图数据库：适合复杂关系，但超出当前技术栈和首版范围。 + +**Consequences**：迁移文件需要严格审查，后续词库更新必须创建新迁移，不能修改已执行迁移。 + +### D3: 三个接口共享同一条排名管道 + +**Choice**：先查询筛选范围内的技能列表并按热度排序，再从同一列表截取 TOP10；匹配和学习路径均调用同一领域服务，不重新排序。 + +**Rationale**：词云、匹配度和学习路径必须对“热门技能集合”保持一致，减少重复逻辑和边界分歧。 + +**Alternatives**： +- 每个接口独立查询和排序：实现分散，容易出现排序不一致。 +- 持久化统一排名快照：可缓存，但当前数据规模不需要额外调度和一致性成本。 + +**Consequences**：排序规则、空热度规则和别名规则只有一个实现位置；单元测试必须聚焦该共享管道。 + +### D4: TOP10 不单独存储或返回 + +**Choice**：词云接口返回完整排序列表，前端直接取前十条；匹配服务在内部最多取前十条。 + +**Rationale**：用户已确认不需要独立 TOP10 数据源，列表本身是唯一事实来源。 + +**Alternatives**：响应同时返回 `top10` 字段或提供独立接口；会重复数据并增加一致性风险。 + +**Consequences**：前端必须按约定截取前十条；接口文档需明确列表已按排名排序。 + +### D5: 匹配和建议按请求实时计算且不持久化 + +**Choice**：每次读取当前学生技能、词库和筛选条件后计算匹配度和学习路径，不保存计算结果。 + +**Rationale**：数据量小、算法确定，且用户未要求历史记录；避免引入无用状态和隐私数据留存。 + +**Alternatives**：保存每日匹配快照或学习计划；会增加表、清理与一致性复杂度，首版无收益。 + +**Consequences**：学生技能或词库更新后，结果立即变化；响应必须包含足够字段让前端独立展示。 + +### D6: 学习建议使用分类模板 + +**Choice**：技能分类映射到编程语言、框架、工具和通用四类建议模板。 + +**Rationale**：满足“技能 + 优先级 + 简化建议”，无需依赖外部课程数据。 + +**Alternatives**：调用 AI 或维护课程表；前者不确定且超出约束，后者缺少可用授权数据。 + +**Consequences**：建议保持简短、行业无关和可重复；后续可在不改变接口结构的前提下细化模板。 + +## Data Model + +| 表 | 关键字段 | 作用 | +| --- | --- | --- | +| `skill_industries` | `id`, `code`, `name_zh`, `source_code`, `sort_order` | 行业节点 | +| `skill_occupations` | `id`, `industry_id`, `code`, `name_zh`, `source_code`, `sort_order` | 行业下职位 | +| `skill_categories` | `id`, `occupation_id`, `code`, `name_zh`, `suggestion_template_code`, `sort_order` | 职位下技能分类 | +| `skills` | `id`, `code`, `name_zh`, `name_en`, `source_type`, `source_code` | 标准技能 | +| `skill_aliases` | `id`, `skill_id`, `alias`, `normalized_alias` | 中英文、大小写和写法别名 | +| `occupation_skills` | `occupation_id`, `skill_id`, `category_id`, `importance`, `importance_source_code`, `importance_element_id`, `importance_is_proxy`, `relationship_type`, `source_version` | 职位技能关系、热度及其机器可追溯来源 | +| `skill_source_versions` | `source_name`, `source_version`, `license`, `attribution_url`, `imported_at` | 来源与许可追溯 | + +约束与索引： + +- `skill_occupations.industry_id + code` 唯一，防止同一行业内职位重复。 +- `skill_categories.occupation_id + code` 唯一。 +- `skill_aliases.normalized_alias` 全局唯一，避免同一写法映射到多个技能。 +- `occupation_skills.occupation_id + skill_id` 唯一。 +- 每条非空 `importance` 必须有 O*NET `importance_source_code` 和 `importance_element_id`；代理映射必须设置 `importance_is_proxy=1`。 +- 查询索引覆盖 `occupation_id + category_id` 和 `occupation_id + importance`。 +- `category_id` 必须属于 `occupation_id`，通过服务校验并在必要测试中验证。 + +## API Contract + +### `GET /api/v1/skills/word-cloud` + +查询参数：`industryId` 必填，`occupationId` 必填，`categoryId` 可选。 + +返回 `data`： + +- `industry`、`occupation`、`category`：实际筛选节点。 +- `total`：技能总数。 +- `items`：按 `heat DESC, nameZh ASC, skillId ASC` 排序，包含 `rank`、`skillId`、`code`、`nameZh`、`nameEn`、`categoryId`、`categoryName`、`heat`、`heatSource`。 + +### `GET /api/v1/skills/match` + +使用与词云相同的三个查询参数。返回 `data`： + +- `matchScore`：0–100 整数。 +- `matchedCount` 与 `totalCount`。 +- `matchedSkills` 与 `missingSkills`：使用精简技能项，缺失技能保持热度顺序。 + +### `GET /api/v1/skills/learning-path` + +使用与词云相同的三个查询参数。返回 `data`： + +- `total`。 +- `items`：每项包含 `priority`、`skillId`、`skillName`、`categoryId`、`categoryName`、`suggestion`。 + +## Processing Rules + +1. Flyway 先建立表和约束，再导入代表性词库数据。 +2. 查询服务校验行业、职位、分类关系；ESCO 作为职位与通用技能主表，O*NET `software_skills` 仅补充 ESCO 缺失的具体技术，构建统一的排序技能列表。 +3. 匹配服务解析当前学生 `skills_json`；解析失败按空技能处理并记录告警，不输出原始 JSON。 +4. 学生技能和技能别名统一执行 Unicode 规范化、去首尾空白、大小写折叠、连续空白折叠及常见标点归一化。 +5. TOP10 大小取 `min(10, 列表大小)`；匹配度按 `round(matched * 100.0 / size)` 计算，`size=0` 时返回 0。 +6. 学习路径遍历缺失技能，按列表顺序生成连续优先级，并依据 `suggestion_template_code` 产生建议。 +7. 所有计算在只读事务中完成，不写入匹配或学习路径记录。 + +## Security and Privacy + +- 在 SecurityFilterChain 中将 `/skills/**` 限制为已认证请求，Controller/Service 再断言 STUDENT 角色。 +- 身份只取 SecurityContext/JWT，不接受 `studentUserId`、`userId` 或 `skills` 请求参数。 +- 响应不包含原始技能 JSON、手机号、邮箱、简历内容或词库内部敏感字段。 +- 日志只记录异常类型与请求上下文，不记录完整学生技能数组。 + +## Risks and Verification + +| 风险 | 处理 | 验证证据 | +| --- | --- | --- | +| ESCO/O*NET 中文覆盖不足 | 首版只导入代表性职位，所有入选技能必须有中文标准名 | 种子数据检查：中文名非空率 100% | +| 别名冲突导致匹配不确定 | `normalized_alias` 全局唯一，迁移失败即阻断 | Flyway 唯一约束测试和别名单测 | +| Importance 缺失 | `heat=null`，排序末位，保留技能 | 排序单元测试和 H2 Repository 测试 | +| 分类关系错配 | 校验分类属于职位、职位属于行业 | 非法组合 MockMvc 400 场景 | +| 迁移过大影响启动 | 首版限制三类行业和代表性职位，种子数据可控 | 迁移在 H2/MySQL 测试库执行耗时记录 | +| 学生技能 JSON 损坏 | 捕获解析异常并按空技能处理 | 异常 JSON 服务测试，断言 200 和匹配度 0 | +| 三个接口结果漂移 | 使用共享排名服务与同一 DTO 排序 | 集成测试断言 TOP10、匹配和学习路径顺序一致 | +| 许可或版本遗漏 | 保存来源版本、许可和归属 URL | 来源说明文档审查与迁移数据断言 | + +## Completion Boundary + +设计完成的标准是：四个规划产物一致、所有需求有可测试场景、任务覆盖三个接口和 Flyway 数据变更、没有未确认的兼容性或权限决策。实现和验收不在本设计阶段启动。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/execution-contract.md b/changes/n03-skill-cloud/execution-contract.md new file mode 100644 index 0000000..20b415d --- /dev/null +++ b/changes/n03-skill-cloud/execution-contract.md @@ -0,0 +1,147 @@ +# N03 行业技能词云与学习路径执行契约 + +## Intent Lock + +**问题**：学生无法从行业和职位维度查看招聘市场热门技能，无法量化自己相对 TOP10 的掌握程度，也没有针对缺失技能的优先级建议。 + +**意图**：基于 ESCO + O*NET 开源知识底座，通过 Flyway 建立可追溯的行业—职位—技能分类—技能数据。ESCO 作为职位、通用技能和分类主表，O*NET `software_skills` 仅补充 ESCO 缺失的具体技术；O*NET Importance 提供热度及机器可追溯来源。系统提供仅 STUDENT 可访问的词云、个人匹配度和学习路径三个确定性接口。 + +**完成证明**：V35/V36 迁移成功执行；三个接口按同一热度列表工作；权限、排序、别名、匹配分母、空数据和通用学习建议均通过自动化测试；OpenAPI 与文档一致。 + +## Scope Fence + +### 允许范围 + +- 新增 `mydreamjob-server/src/main/resources/db/migration/V35__skill_cloud_taxonomy.sql` 和 `V36__skill_cloud_seed.sql`。 +- 新增 `com.mydreamjob.skill` 领域包、Repository、Service、DTO 和三个 Controller。 +- 修改安全配置以保护 `/skills/**`，仅允许已登录 STUDENT。 +- 新增 ESCO/O*NET 来源说明、接口文档、OpenAPI 契约和测试。\n- 新增从 O*NET 31.0 官方 CSV 提取的测试证据快照，仅用于全量核对种子来源和 Importance，不参与运行时。 +- 首版数据只覆盖互联网、金融、教育及代表性职位。 +- 具体技术无可靠 ESCO 节点时，允许使用 O*NET `software_skills` 作为补充来源。 +- 词云热度只取 O*NET Importance；无映射时 `heat=null`。 +- 匹配度使用当前 TOP10 和实际技能数；学习路径只返回缺失技能、优先级和通用建议。 + +### 禁止范围 + +- 不修改 V1–V34 历史迁移，不改变已有接口语义。 +- 不接 AI、课程平台、第三方运行时接口或在线词库同步。 +- 不按平台岗位 `jobId`、发布量或用户行为计算热度。 +- 不实现后台词库维护、学习进度、提醒、历史快照或前端页面。 +- 不向 COMPANY/ADMIN 返回个人匹配或学习路径。 +- 不在响应或日志中输出原始 `skills_json`、手机号、邮箱、简历内容。 + +## Approved Behavior + +| ID | 已批准行为 | 批次 | +| --- | --- | --- | +| REQ-SC-1 | `/skills/word-cloud`、`/skills/match`、`/skills/learning-path` 只允许已登录 STUDENT；匿名返回 401，COMPANY/ADMIN 返回 403。 | B3 | +| REQ-SC-2 | `industryId`、`occupationId` 必填且必须互为关联；`categoryId` 可选，存在时必须属于目标职位；非法组合返回 400。 | B2、B3 | +| REQ-SC-3 | 词云返回筛选范围内全部技能，按 `heat DESC, nameZh ASC, skillId ASC` 排序；`heat=null` 排在有值技能之后。 | B2 | +| REQ-SC-4 | TOP10 不单独存储或返回，前端取词云前十条；少于十条时全部结果构成实际 TOP10。 | B2、B3 | +| REQ-SC-5 | 每个技能项包含技能 ID、名称、分类、`heat`、稳定排名和可追溯来源标识；响应使用 `ApiResponse`。 | B2、B3 | +| REQ-SM-1 | 个人匹配只使用当前筛选结果的 `min(10, actualSize)` 个技能，不另建排序。 | B2 | +| REQ-SM-2 | 学生技能只从 JWT 对应学生的 `student_profiles.skills_json` 读取；缺失、空数组或损坏 JSON 按无技能处理。 | B2 | +| REQ-SM-3 | 匹配使用标准技能和别名，执行 NFKC、首尾空白、大小写、连字符转空格、连续空白折叠和句点移除归一化；`+`、`#` 不得删除。 | B2 | +| REQ-SM-4 | 匹配度为 `round(matched * 100 / min(10, actualSize))`；`actualSize=0` 时返回 0；重复命中同一技能只计一次。 | B2 | +| REQ-SM-5 | 响应分别返回命中与缺失技能，缺失列表保持 TOP10 顺序，两者并集等于实际 TOP10。 | B2、B3 | +| REQ-LP-1 | 学习路径只包含当前 TOP10 的未命中技能，不包含已命中或 TOP10 之外的技能。 | B2 | +| REQ-LP-2 | 学习路径沿用词云顺序，优先级从 1 连续编号；空热度技能仍在有权重技能之后。 | B2 | +| REQ-LP-3 | 学习建议由编程语言、框架、工具、通用四类确定性模板生成；不调用 AI 或外部服务，不含课程链接、时长或价格。 | B2 | +| REQ-LP-4 | 每项只返回技能 ID、名称、分类、优先级和简化建议；无缺口时返回空列表和总数 0。 | B2、B3 | + +需求覆盖结果：三份 spec 中全部 14 条 `MUST`/`SHALL` 行为均已映射，无未映射需求。 + +## Architecture and Interface Constraints + +- 使用 ESCO 作为职位、通用技能和分类主关系；具体技术无可靠 ESCO 节点时，允许使用 O*NET `software_skills` 作为补充技能来源。 +- O*NET Importance 作为 `heat`；项目维护中文名称、别名和细粒度分类。 +- 数据模型固定为 `skill_industries`、`skill_occupations`、`skill_categories`、`skills`、`skill_aliases`、`occupation_skills`、`skill_source_versions` 七张表。 +- `skill_aliases.normalized_alias` 全局唯一；`occupation_skills(occupation_id, skill_id)` 唯一；分类必须属于职位，职位必须属于行业。 +- `occupation_skills` 必须保存 `importance_source_code`、`importance_element_id` 和 `importance_is_proxy`；非空 `importance` 必须可追溯到 O*NET 31.0 CSV。 +- 三个接口共享同一个只读排名管道和技能投影；不得复制三套排序规则。 +- 接口路径固定为： + - `GET /api/v1/skills/word-cloud` + - `GET /api/v1/skills/match` + - `GET /api/v1/skills/learning-path` +- 三个接口查询参数均为 `industryId`、`occupationId` 必填，`categoryId` 可选。 +- 计算按请求实时完成，不持久化匹配或学习路径。 +- 学生技能解析异常记录不含原始内容的告警，接口返回 200 和空技能匹配结果。 +- 首版只使用代表性数据；迁移编号以当前仓库下一个可用版本为准。若合并前 V35/V36 已被其他变更占用，必须回退规划阶段刷新编号和契约。 + +## Execution Batches + +### Batch B1: 数据与来源基础 + +- 覆盖任务：T1、T2、T3，T3 在第 1 次评审后执行来源与分类修复。 +- 交付物：来源清单、`V35__skill_cloud_taxonomy.sql`、`V36__skill_cloud_seed.sql`、迁移测试。 +- 完成定义：代表性数据包含三类行业和约定职位；所有入选技能有中文标准名；ESCO 职位来源、O*NET 补充技术、每个 Importance 的 SOC/Element ID 均可机器追溯；自动化测试通过版本化证据快照全量核对 O*NET 来源和数值。 +- 依赖：无。 +- 评审门：B1 完成并由 `code-reviewer` 记录 `pass` 后才能开始 B2。 + +### Batch B2: 排名、匹配与学习路径 + +- 覆盖任务：T4、T5、T6、T7。 +- 交付物：实体/Repository、共享词云查询排名服务、别名归一化、个人匹配服务、学习路径模板服务及单元测试。 +- 完成定义：REQ-SC-2 至 REQ-SC-5、REQ-SM-1 至 REQ-SM-5、REQ-LP-1 至 REQ-LP-4 的领域行为可重复且确定。 +- 依赖：B1 通过评审。 +- 评审门：B2 完成并由 `code-reviewer` 记录 `pass` 后才能开始 B3。 + +### Batch B3: 接口、权限、契约与回归 + +- 覆盖任务：T8、T9、T10。 +- 交付物：三个 Controller、权限配置、MockMvc/H2 集成测试、`docs/skill-cloud-api.md`、OpenAPI 和前端联调文档更新。 +- 完成定义：REQ-SC-1 至 REQ-SC-5、REQ-SM-5、REQ-LP-4 的接口行为通过；匿名 401、非 STUDENT 403、非法参数 400；文档与运行时一致。 +- 依赖：B2 通过评审。 +- 评审门：B3 完成并由 `code-reviewer` 记录 `pass` 后才能进入验证和收尾。 + +## Test Obligations + +| ID | 测试义务 | 批次 | 证据 | +| --- | --- | --- | --- | +| TO-1 | Flyway 创建七张表、外键、唯一约束和索引，中文名与来源 ID 完整。 | B1 | `mvn -Dtest=SkillTaxonomyMigrationTest test` | +| TO-2 | 行业/职位/分类关系校验，非法组合返回 400。 | B2、B3 | Repository/Service 单测与 MockMvc | +| TO-3 | 热度降序、空热度末位、同热度稳定排序、空范围列表。 | B2 | `mvn -Dtest=SkillCloudQueryServiceTest test` | +| TO-4 | TOP10 少于十条时使用实际数量，匹配分母为实际数量，`1/3` 四舍五入为 33。 | B2 | `mvn -Dtest=SkillMatchServiceTest test` | +| TO-5 | 大小写、空格、连字符、句点、`C++`、`C#` 和重复别名匹配。 | B2 | 归一化与匹配单测 | +| TO-6 | 无技能、无资料、损坏 JSON、全部命中和全部未命中场景。 | B2、B3 | Service 与 MockMvc 测试 | +| TO-7 | 学习路径只含未命中技能，优先级连续，无缺口返回空列表。 | B2 | `mvn -Dtest=SkillLearningPathServiceTest test` | +| TO-8 | 四类建议模板稳定、行业无关，不含链接、时长、价格或 AI 内容。 | B2 | 模板单测 | +| TO-9 | 三个接口权限、响应结构和参数错误符合契约。 | B3 | `mvn -Dtest=SkillCloudControllerTest test` | +| TO-10 | 三个接口复用同一 TOP10，响应不含原始 `skills_json`。 | B3 | H2 集成测试 | +| TO-11 | 全量回归、打包和静态 OpenAPI 契约通过。 | B3 | `mvn test`、`mvn clean verify`、`python docs\verify_openapi.py` |\n| TO-12 | 测试读取 O*NET 31.0 证据快照，全量核对 47 条 O*NET 技术来源及全部非空 Importance；任一来源不存在或数值被篡改都必须失败。 | B1 | `mvn -Dtest=SkillCloudSeedDataIntegrationTest test` | + +## Build Rules + +- 每个 Batch 按 TDD 先写失败测试，再实现最小行为，不得跳过测试。 +- 只修改范围内文件；发现需要修改已验收接口或历史迁移时立即停止并触发 Rewind。 +- 迁移和种子数据必须可在 H2（`MODE=MySQL`）与真实 MySQL 执行，不依赖外部文件路径。 +- 所有排序、匹配和建议必须确定性；禁止随机、当前时间或请求次数影响结果。 +- 任何新增日志不得包含个人技能 JSON、Token、手机号、邮箱或简历文本。 +- OpenAPI 注解、示例和实际响应必须一致，不能只改文档不改运行时行为。 + +## Review Gates + +- **DP-3**：本契约必须由用户明确批准并记录后，才能开始任何实现。 +- **Batch Review**：B1、B2、B3 每个 Batch 完成后由 `code-reviewer` 按规范符合性和代码质量评审，记录 `pass` 或 `fail`。 +- **DP-4**：批准契约后由 `build-executor` 给出执行模式与 wave 计划，用户确认后开始执行。 +- **Verification Gate**：实现完成后执行全量测试、OpenAPI 校验和必要的 MySQL/Docker 黑盒验证。 +- **DP-7**：由 `release-archivist` 完成最终验证与关闭，不在本契约内自动完成。 + +## Rewind Conditions + +出现以下任一情况必须停止执行并回到规划阶段刷新工件和本契约： + +- 需要新增、删除或改变用户可见接口、筛选参数、权限或响应语义。 +- TOP10、匹配度分母、排序规则或空数据处理发生改变。 +- 需要引入 AI、课程平台、实时岗位热度或后台维护接口。 +- 需要导入全量词库或扩大首版行业/职位范围。 +- 需要新增除 ESCO、O*NET 之外的其他技能来源。 +- V35/V36 被其他变更占用，或历史迁移需要修改。 +- ESCO/O*NET 许可、中文映射策略或数据来源发生实质变化。 +- 发现某个已批准 `MUST`/`SHALL` 无法映射到测试或执行批次。 + +## DP-3 Approval Gate + +执行者确认：本契约完整覆盖已批准的 14 条必需行为，无未映射需求，B1→B2→B3 依赖清晰，所有测试义务均有证据命令。 + +**批准方式**：用户明确回复批准本执行契约后，记录 `dp_3_result`、`dp_3_timestamp`，再执行 `specifying → bridging → approved-for-build` 状态转换。DP-3 记录前禁止进入 `build-executor`。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/proposal.md b/changes/n03-skill-cloud/proposal.md new file mode 100644 index 0000000..930da97 --- /dev/null +++ b/changes/n03-skill-cloud/proposal.md @@ -0,0 +1,59 @@ +# N03 行业技能词云与学习路径接口 + +## Why + +当前系统已经具备学生资料、技能标签、岗位匹配和求职进度能力，但学生无法从行业与职位的视角看到“市场在要求哪些技能”，也无法知道自己相对热门技能 TOP10 的掌握程度，更无法直接得到按优先级排列的学习补缺清单。仓库截至 V34 没有技能词云、行业技能热度或个人学习路径实现，现有 `student_profiles.skills_json` 与 `jobs.requirements` 也尚未被统一为可查询的技能分类体系。 + +本变更通过引入 ESCO + O*NET 的开源知识底座，为“行业 → 职位 → 技能分类 → 技能”建立可版本化数据，并为已登录学生提供三个确定性接口：查看词云与 TOP10、计算个人技能匹配度、生成缺失技能学习路径。 + +## What Changes + +- 新增 Flyway 词库表，保存行业、职位、技能分类、技能、别名、职位技能关系、O*NET Importance 热度及数据来源版本。 +- 首版导入互联网、金融、教育行业及代表性职位；中文名称、别名和细粒度技能分类由项目维护映射。 +- 新增 `GET /api/v1/skills/word-cloud`：按行业、职位、可选技能分类返回全部技能，按热度降序排列。 +- 新增 `GET /api/v1/skills/match`：用当前学生技能命中当前筛选结果 TOP10，返回匹配度、命中技能和缺失技能。 +- 新增 `GET /api/v1/skills/learning-path`：对缺失 TOP10 技能按热度生成优先级和通用学习建议。 +- 三个接口仅允许已登录 STUDENT 访问，统一使用现有 JWT、角色校验和 `ApiResponse` 规范。 +- 更新 OpenAPI 契约、前端联调文档和词库来源说明。 + +## Scope + +### In Scope + +- ESCO 与 O*NET 的代表性数据导入、中文映射、来源版本和许可归属。 +- 行业、职位、技能分类、技能、别名、热度及职位技能关系查询。 +- 当前筛选范围内的完整技能词云，以及可由前端直接截取的 TOP10。 +- 个人 `skills_json` 与 TOP10 的别名归一化匹配。 +- 匹配度按 `命中数 / min(10, 实际技能数)` 计算，实际技能数为 0 时返回 0。 +- 缺失技能按同一热度排序生成优先级 1、2、3…。 +- 按技能分类生成通用学习建议，不依赖 AI 或外部服务。 +- 单元测试、MockMvc 测试、H2 持久化测试和 OpenAPI 静态校验。 + +### Out of Scope + +- 平台岗位 `jobId`、岗位发布频率或实时文本抽取参与热度计算。 +- ESCO/O*NET 全量导入、自动翻译全量技能名称、在线同步开源词库。 +- 后台词库维护页面、运营导入工具或运行时更新接口。 +- AI 生成课程大纲、课程链接、讲师、学习时长或培训平台对接。 +- 保存个人匹配历史、学习进度、完成状态或提醒任务。 +- 企业和管理员查看学生个人匹配结果。 +- 前端页面、交互设计和可视化实现。 + +## Impact + +- `mydreamjob-server/src/main/resources/db/migration`：新增词库结构迁移和代表性种子数据。 +- `mydreamjob-server/src/main/java/com/mydreamjob`：新增技能词云领域包、实体、Repository、Service、DTO 和 Controller，并调整安全配置。 +- `mydreamjob-server/src/test/java/com/mydreamjob`：新增查询排序、匹配、学习路径和权限场景测试。 +- `docs/openapi-rc.json`、`docs/frontend-integration.md`：登记三个接口和响应示例。 +- 新增词库来源与版本说明文档，保留 ESCO/O*NET 的许可和数据版本。 +- 不修改已有岗位匹配、候选人评分、求职进度等已验收接口语义。 + +## Proof of Completion + +- Flyway 在 MySQL 模式测试库成功创建词库表并导入代表性数据。 +- 三个接口返回统一 `ApiResponse`，匿名访问为 401，非 STUDENT 为 403。 +- 词云结果按 O*NET Importance 降序；无权重技能保留且排在有权重技能之后；同权重排序稳定。 +- TOP10 与词云使用同一排序结果，匹配度和学习路径复用该结果。 +- 匹配度覆盖 0、少于 10 项、恰好 10 项、全部命中和部分命中场景。 +- 学习路径优先级连续，且只包含未命中的 TOP10 技能；无缺口时返回空列表。 +- `mvn test`、`mvn clean verify` 和 `python docs/verify_openapi.py` 全部通过。 \ No newline at end of file diff --git a/changes/n03-skill-cloud/specs/learning-path/spec.md b/changes/n03-skill-cloud/specs/learning-path/spec.md new file mode 100644 index 0000000..f95e8e5 --- /dev/null +++ b/changes/n03-skill-cloud/specs/learning-path/spec.md @@ -0,0 +1,84 @@ +# Learning Path Specification + +## ADDED Requirements + +### Requirement: Learning path consumes unmatched TOP10 skills + +学习路径接口 MUST 复用个人技能匹配结果，并只包含当前 TOP10 中未命中的技能；不得包含已命中技能或 TOP10 之外的技能。 + +#### Scenario: Student has skill gaps + +- **WHEN** 当前 TOP10 中部分技能未命中 +- **THEN** 学习路径 MUST 包含全部未命中技能且不含已命中技能 + +#### Scenario: Student has no skill gaps + +- **WHEN** 当前 TOP10 全部命中 +- **THEN** 学习路径 MUST 返回空列表和总数 0 + +#### Scenario: Student has no matching skill evidence + +- **WHEN** 学生没有技能数据 +- **THEN** 学习路径 MUST 包含当前 TOP10 的全部技能 + +### Requirement: Learning priority follows heat order + +学习路径项 MUST 按与 TOP10 相同的热度顺序排列，并从 1 开始连续编号；空热度技能继续遵守词云的末位排序规则。 + +#### Scenario: Multiple missing skills are ordered + +- **WHEN** 缺失技能包含多个热度不同的技能 +- **THEN** 热度最高的技能 MUST 获得优先级 1，后续技能依次编号 + +#### Scenario: Missing skills have equal heat + +- **WHEN** 缺失技能热度相同 +- **THEN** 优先级 MUST 沿用词云的稳定次级排序，重复请求结果一致 + +#### Scenario: Only one skill is missing + +- **WHEN** 当前 TOP10 只有一项未命中 +- **THEN** 该技能 MUST 返回优先级 1 + +### Requirement: Learning suggestions are deterministic and industry-neutral + +每项学习建议 MUST 由后端按技能分类的通用模板生成，不调用 AI、知识库问答或外部课程服务；建议不得包含课程链接、讲师、价格、学习时长或完成度。 + +#### Scenario: Programming language gap is returned + +- **WHEN** 缺失技能属于编程语言分类 +- **THEN** 建议 MUST 描述基础语法学习和编码练习方向 + +#### Scenario: Framework gap is returned + +- **WHEN** 缺失技能属于框架分类 +- **THEN** 建议 MUST 描述核心概念学习和小型项目实践方向 + +#### Scenario: Tool gap is returned + +- **WHEN** 缺失技能属于工具分类 +- **THEN** 建议 MUST 描述安装配置和工程实践方向 + +#### Scenario: Unknown category is returned + +- **WHEN** 缺失技能无法匹配特定模板分类 +- **THEN** 系统 MUST 返回一条行业无关的通用学习建议 + +### Requirement: Learning path response is actionable and minimal + +每个学习路径项 MUST 返回技能 ID、技能名称、技能分类、优先级和简化学习建议；响应 MUST 不包含个人隐私、简历原文或模型推理内容。 + +#### Scenario: A learning item is generated + +- **WHEN** 存在未命中技能 +- **THEN** 每个路径项 MUST 同时具备技能身份、分类、连续优先级和建议文本 + +#### Scenario: Repeated request is made + +- **WHEN** 学生资料、词库数据和筛选条件均未变化 +- **THEN** 接口 MUST 返回相同顺序、相同优先级和相同建议 + +#### Scenario: Response contains no private profile data + +- **WHEN** 学习路径成功返回 +- **THEN** 响应 MUST 不含手机号、邮箱、简历文件、原始 `skills_json` 或其他无关个人数据 \ No newline at end of file diff --git a/changes/n03-skill-cloud/specs/skill-cloud/spec.md b/changes/n03-skill-cloud/specs/skill-cloud/spec.md new file mode 100644 index 0000000..f601c13 --- /dev/null +++ b/changes/n03-skill-cloud/specs/skill-cloud/spec.md @@ -0,0 +1,98 @@ +# Skill Cloud Specification + +## ADDED Requirements + +### Requirement: STUDENT-only skill cloud access + +系统 MUST 只允许已登录 STUDENT 调用技能词云接口，并从 JWT 解析当前用户，不接受客户端传入学生身份参数。 + +#### Scenario: Anonymous request is rejected + +- **WHEN** 未携带有效 JWT 请求 `GET /api/v1/skills/word-cloud` +- **THEN** 系统 MUST 返回 401，且不得返回任何词库数据 + +#### Scenario: Company or administrator request is rejected + +- **WHEN** COMPANY 或 ADMIN 用户携带有效 JWT 请求技能词云 +- **THEN** 系统 MUST 返回 403，且不得返回任何词库数据 + +#### Scenario: Student request is accepted + +- **WHEN** STUDENT 用户携带有效 JWT 请求技能词云 +- **THEN** 系统 MUST 按筛选条件返回词库数据 + +### Requirement: Skill cloud filter validation + +接口 MUST 要求 `industryId` 和 `occupationId` 为必填正整数，`categoryId` 为可选正整数；行业与职位必须存在且互为关联，技能分类存在时必须属于该职位。 + +#### Scenario: Required filters are missing + +- **WHEN** 请求未传 `industryId` 或 `occupationId` +- **THEN** 系统 MUST 返回 400，并说明缺失的筛选参数 + +#### Scenario: Occupation does not belong to industry + +- **WHEN** 请求传入存在但不属于目标行业的职位 +- **THEN** 系统 MUST 返回 400，且不得返回技能数据 + +#### Scenario: Category does not belong to occupation + +- **WHEN** 请求传入存在但不属于目标职位的技能分类 +- **THEN** 系统 MUST 返回 400，且不得返回技能数据 + +#### Scenario: Category is omitted + +- **WHEN** 请求只传行业和职位 +- **THEN** 系统 MUST 返回该职位下所有技能分类的技能 + +### Requirement: Complete ranked skill cloud + +系统 MUST 返回当前筛选范围内全部技能，并按 `heat` 降序排列；`heat` 为空时排在非空值之后。同 `heat` 或同为空值时，系统 MUST 使用稳定的中文技能名称和技能 ID 作为次级排序条件。 + +#### Scenario: Skills are ordered by heat + +- **WHEN** 筛选结果存在多个带 O*NET Importance 的技能 +- **THEN** 返回列表 MUST 按数值从高到低排列 + +#### Scenario: Skills without heat are retained + +- **WHEN** 筛选结果包含无 O*NET Importance 的 ESCO 技能 +- **THEN** 这些技能 MUST 保留在列表中，`heat` 为 null，并排在所有非 null 热度技能之后 + +#### Scenario: Equal heat has deterministic order + +- **WHEN** 两个技能具有相同 `heat` +- **THEN** 系统 MUST 按稳定次级规则返回一致顺序，重复请求不得随机变化 + +#### Scenario: Empty scope returns empty list + +- **WHEN** 合法筛选范围内没有技能 +- **THEN** 系统 MUST 返回成功响应、空技能列表和总数 0 + +### Requirement: TOP10 is derived from the skill cloud + +系统 MUST NOT 为 TOP10 维护独立排序或独立数据源；前端必须能够直接使用词云返回列表的前十条得到 TOP10。 + +#### Scenario: Scope has at least ten skills + +- **WHEN** 词云结果包含至少十条技能 +- **THEN** 前十条 MUST 与同一筛选条件下的热门技能 TOP10 一致 + +#### Scenario: Scope has fewer than ten skills + +- **WHEN** 词云结果少于十条技能 +- **THEN** 全部结果 MUST 构成实际 TOP10 集合 + +### Requirement: Skill cloud response exposes source-linked metadata + +每个技能项 MUST 至少返回技能 ID、中文名称、技能分类 ID、技能分类名称、`heat` 和稳定排名。技能仍可通过 ID 追溯至 ESCO/O*NET 来源。 + +#### Scenario: Skill item is returned + +- **WHEN** 查询返回任一技能 +- **THEN** 响应 MUST 包含该技能的身份、分类、热度和排名字段 + +#### Scenario: Response is wrapped consistently + +- **WHEN** 请求成功 +- **THEN** 响应 MUST 使用项目统一 `ApiResponse` 结构，并在 `data` 中返回筛选条件和技能列表 \ No newline at end of file diff --git a/changes/n03-skill-cloud/specs/skill-match/spec.md b/changes/n03-skill-cloud/specs/skill-match/spec.md new file mode 100644 index 0000000..2b9ff04 --- /dev/null +++ b/changes/n03-skill-cloud/specs/skill-match/spec.md @@ -0,0 +1,98 @@ +# Skill Match Specification + +## ADDED Requirements + +### Requirement: Personal matching uses the current ranked TOP10 + +个人技能匹配接口 MUST 复用技能词云相同的筛选参数、数据范围和热度排序；匹配集合 MUST 是当前筛选结果的前 `min(10, 实际技能数)` 项。 + +#### Scenario: Filtered result has at least ten skills + +- **WHEN** 当前筛选返回十条或更多技能 +- **THEN** 匹配集合 MUST 恰好包含热度最高的十条技能 + +#### Scenario: Filtered result has fewer than ten skills + +- **WHEN** 当前筛选返回六条技能 +- **THEN** 匹配集合 MUST 包含全部六条技能，分母 MUST 为 6 + +#### Scenario: Filtered result is empty + +- **WHEN** 当前合法筛选范围内没有技能 +- **THEN** 匹配度 MUST 返回 0，命中和缺失列表 MUST 均为空 + +### Requirement: Student skills are read from the authenticated profile + +系统 MUST 从当前登录 STUDENT 的 `student_profiles.skills_json` 读取技能；不得接受客户端另行提交的技能列表作为个人匹配依据。 + +#### Scenario: Student profile has valid skills + +- **WHEN** 当前学生存在合法技能数组 +- **THEN** 系统 MUST 使用该数组计算匹配结果 + +#### Scenario: Student profile is absent or has no skills + +- **WHEN** 当前学生没有资料记录、技能字段为空或技能数组为空 +- **THEN** 匹配度 MUST 返回 0，命中列表为空，缺失列表为当前 TOP10 + +#### Scenario: Skills JSON is malformed + +- **WHEN** 当前学生技能字段无法解析为字符串数组 +- **THEN** 系统 MUST 按无技能处理，不得导致 500 响应或泄露原始字段内容 + +### Requirement: Skill matching normalizes aliases + +系统 MUST 使用技能标准和别名进行匹配，归一化至少覆盖首尾空白、大小写、连续空格和常见标点差异；每个标准技能命中一次即可。 + +#### Scenario: Alias matches canonical skill + +- **WHEN** 学生技能为 `SpringBoot`，TOP10 标准技能为 `Spring Boot` 且存在对应别名 +- **THEN** 系统 MUST 将该技能计为命中 + +#### Scenario: Case and spacing differ + +- **WHEN** 学生技能与标准技能的字母大小写或空格数量不同 +- **THEN** 系统 MUST 在归一化后判断为同一技能 + +#### Scenario: Duplicate student skills exist + +- **WHEN** 学生技能数组包含多个指向同一标准技能的别名 +- **THEN** 系统 MUST 只计一次命中 + +### Requirement: Match score uses the actual TOP10 size + +匹配度 MUST 使用公式 `round(命中数 × 100 / min(10, 实际技能数))` 计算，并返回 0 到 100 的整数。实际技能数为 0 时，匹配度 MUST 为 0。 + +#### Scenario: All skills match + +- **WHEN** 当前 TOP10 有 8 项，学生命中全部 8 项 +- **THEN** 匹配度 MUST 返回 100 + +#### Scenario: Partial match is rounded + +- **WHEN** 当前 TOP10 有 3 项，学生命中 1 项 +- **THEN** 匹配度 MUST 返回 33 + +#### Scenario: No skill matches + +- **WHEN** 当前 TOP10 非空且学生没有命中任何技能 +- **THEN** 匹配度 MUST 返回 0 + +### Requirement: Match response separates matched and missing skills + +响应 MUST 返回匹配度、命中数量、实际匹配集合大小、命中技能列表和缺失技能列表；缺失技能 MUST 保持 TOP10 的热度排序。 + +#### Scenario: Some skills are matched + +- **WHEN** 学生命中 TOP10 的部分技能 +- **THEN** 命中列表和缺失列表 MUST 分别返回对应技能，且并集等于 TOP10 + +#### Scenario: All skills are matched + +- **WHEN** 学生命中 TOP10 的全部技能 +- **THEN** 缺失技能列表 MUST 为空，匹配度 MUST 为 100 + +#### Scenario: No skills are matched + +- **WHEN** 学生没有命中 TOP10 的任何技能 +- **THEN** 命中技能列表 MUST 为空，缺失列表 MUST 等于 TOP10 \ No newline at end of file diff --git a/changes/n03-skill-cloud/tasks.md b/changes/n03-skill-cloud/tasks.md new file mode 100644 index 0000000..67ff6cb --- /dev/null +++ b/changes/n03-skill-cloud/tasks.md @@ -0,0 +1,38 @@ +# 实施任务 + +## 交付与证明映射 + +| 需求域 | 规划来源 | 实现任务 | 主要证明 | +| --- | --- | --- | --- | +| 词库来源与覆盖 | proposal、design | T1、T2、T3 | 来源清单、中文名检查、Flyway 执行成功 | +| 词云筛选与排序 | skill-cloud spec | T4、T5 | Repository/Service 单测、MockMvc 场景 | +| TOP10 派生与一致性 | skill-cloud、skill-match spec | T5、T7、T8 | 集成测试断言同一排序结果 | +| 个人技能匹配 | skill-match spec | T6、T8 | 别名、分母、空数据和部分命中单测 | +| 学习路径与建议 | learning-path spec | T7、T8 | 优先级连续、模板建议和空列表测试 | +| 权限与隐私 | 三份 spec、design | T8、T10 | 401/403、响应脱敏和日志检查 | +| 文档与契约 | proposal、design | T1、T10 | `docs/verify_openapi.py`、接口文档审查 | + +## 任务 + +- [ ] 1. 固化 ESCO + O*NET 代表性数据清单：在 `changes/n03-skill-cloud/data/source-manifest.md` 和 `docs/skill-taxonomy-sources.md` 记录行业、职位、技能范围、源版本、许可、归属 URL、中文映射负责人和导入规则。产出：互联网、金融、教育及代表性职位的可审查数据边界。依赖：无。证据：人工审查清单，所有入选技能均有稳定来源 ID 和中文标准名。 +- [ ] 2. 新增 Flyway 词库结构迁移：创建 `mydreamjob-server/src/main/resources/db/migration/V35__skill_cloud_taxonomy.sql`，实现七张词库表、外键、唯一约束、查询索引，以及 `occupation_skills.importance_source_code`、`importance_element_id`、`importance_is_proxy` 来源追溯字段，不修改 V1–V34。产出：H2/MySQL 均可创建词库结构。依赖：T1。证据：`mvn -Dtest=SkillTaxonomyMigrationTest test`。 +- [ ] 3. 导入代表性种子数据：创建 `mydreamjob-server/src/main/resources/db/migration/V36__skill_cloud_seed.sql`，导入三类行业、代表性职位、全部已声明技能分类、ESCO 主来源技能、O*NET `software_skills` 补充技术、别名和 O*NET Importance；每条非空热度必须保存真实 SOC、Element ID 和代理标记，无映射时 `importance` 留空。产出：种子数据可重复执行迁移并满足中文名约束。依赖：T2。证据：迁移测试断言数量、中文名、分类覆盖、ESCO 职位来源、47 条 O*NET 技术来源和全部非空 Importance 与版本化证据快照一致。 +- [ ] 4. 建立词库领域与查询仓储：在 `mydreamjob-server/src/main/java/com/mydreamjob/skill/` 新增实体、Repository 和查询投影，覆盖行业—职位—分类校验及技能关系查询。产出：能按 `industryId`、`occupationId`、可选 `categoryId` 读取技能。依赖：T2、T3。证据：`mvn -Dtest=SkillTaxonomyRepositoryTest test`。 +- [ ] 5. 实现共享词云排名服务与 DTO：在 `mydreamjob-server/src/main/java/com/mydreamjob/skill/` 实现统一筛选、`heat DESC, nameZh ASC, skillId ASC` 排序和完整列表输出；返回 `rank`、技能身份、分类、`heat`、来源字段。产出：词云和 TOP10 使用同一个事实来源。依赖：T4。证据：`mvn -Dtest=SkillCloudQueryServiceTest test`，覆盖空热度、同热度、空范围和分类缺省。 +- [ ] 6. 实现个人技能匹配服务：新增 `SkillMatchService`、别名归一化器和匹配 DTO，只从 JWT 对应学生的 `skills_json` 读取技能；按 `min(10, actualSize)` 截取并计算整数匹配度。产出：命中、缺失、分母和异常 JSON 行为符合规范。依赖：T5。证据：`mvn -Dtest=SkillMatchServiceTest test`，覆盖 0、1/3=33、全命中、少技能和重复别名。 +- [ ] 7. 实现学习路径生成服务：新增 `SkillLearningPathService` 和分类模板，按缺失技能顺序生成连续优先级与通用建议；不接 AI 或外部资源。产出：无缺口返回空列表，有缺口返回最小学路径。依赖：T6。证据：`mvn -Dtest=SkillLearningPathServiceTest test`。 +- [ ] 8. 暴露三个 STUDENT 接口并登记权限：新增 `SkillCloudController`，实现 `GET /skills/word-cloud`、`GET /skills/match`、`GET /skills/learning-path`，在 `SecurityConfig` 中加入 `/skills/**` 的登录与 STUDENT 校验，并补充 OpenAPI 注解。产出：三个 API 只接受 STUDENT，响应统一为 `ApiResponse`。依赖：T5、T6、T7。证据：`mvn -Dtest=SkillCloudControllerTest test`，覆盖匿名 401、COMPANY/ADMIN 403、参数非法 400 和学生成功 200。 +- [ ] 9. 补齐持久化与端到端回归测试：在 `mydreamjob-server/src/test/java/com/mydreamjob/skill/` 增加 H2 持久化、排序一致性、匹配与学习路径复用同一 TOP10 的集成测试，并验证响应不含原始 `skills_json`。产出：三个接口的边界与一致性有自动化证明。依赖：T3、T8。证据：`mvn test`。 +- [ ] 10. 更新接口与来源文档并执行发布级验证：新增 `docs/skill-cloud-api.md`，更新 `docs/frontend-integration.md`、`docs/openapi-rc.json` 和词库来源说明，明确 TOP10 由前端截取、参数规则、响应字段、无 AI 和许可归属。依赖：T8、T9。产出：静态契约、运行时行为和文档一致。证据：`python docs\verify_openapi.py`、`mvn clean verify`。 + +## 实施顺序 + +T1 → T2 → T3 → T4 → T5 → T6 → T7 → T8 → T9 → T10。 + +## 退出条件 + +- 所有任务完成且证据命令通过。 +- 三份 spec 的场景全部有对应自动化或明确人工契约验证。 +- 词库来源版本和许可可追溯，未修改历史迁移。 +- 未实现非目标范围；没有课程、AI、后台维护或个人进度持久化。 +- OpenAPI 静态契约与运行时接口一致，且 STUDENT 权限边界可验证。 \ No newline at end of file diff --git a/docs/frontend-integration.md b/docs/frontend-integration.md index 599cdf3..d4b397e 100644 --- a/docs/frontend-integration.md +++ b/docs/frontend-integration.md @@ -715,3 +715,22 @@ GET /profile/me/career-progress?page=0&size=20&status=INTERVIEW # 仅 STUDENT - **脱敏**：不返回 `studentUserId/candidateId/companyUserId/evaluatorUserId/phone/email/resumeUrl/passwordHash/JWT`、通知标题/正文、评分 `comment`、Offer 薪资/条款、他人任何身份字段。**本接口只返回通知计数，通知详情仍用 `/notifications`**。 - **只读**：`GET`，不写任何业务表；`size` 上限 50，聚合查询次数与单页条数无关（无 N+1）。 - API 详见 [docs/career-progress-api.md](career-progress-api.md)。 + +## 21. 行业技能词云、个人匹配度与学习路径联调 + +三个接口共用行业、职位和技能分类筛选，仅 STUDENT 可调用，并统一使用 Bearer JWT。完整字段、错误示例和排序语义详见 [skill-cloud-api.md](skill-cloud-api.md)。 + +| 功能 | Method | Path | 关键返回/交互 | +|---|---|---|---| +| 行业技能词云 | `GET` | `/skills/word-cloud` | `data.items` 为完整排序结果；前端直接取前 10 条作为 TOP10。 | +| 个人技能匹配度 | `GET` | `/skills/match` | 服务端只读取 JWT 学生资料；返回匹配度、命中技能和缺失技能。 | +| 学习路径 | `GET` | `/skills/learning-path` | 只返回缺失技能、连续优先级和通用学习建议。 | + +共同 query 参数：`industryId`、`occupationId` 必填正整数；`categoryId` 可选正整数。非法关系返回 `400`，未登录 `401`，非 STUDENT `403`。 + +联调注意： + +- `heat=null` 的技能仍会返回，且必须排在所有非空 `heat` 之后；不要在前端丢弃这些技能或自行改变顺序。 +- TOP10 不是独立接口字段；统一使用 `data.items.slice(0, min(10, data.total))`。 +- 匹配度公式为 `round(matchedCount * 100 / min(10, total))`；范围 15 条、命中 5 条时为 `33`。 +- 学习路径不返回课程链接、AI 文本、学习时长或进度记录。 diff --git a/docs/openapi-rc.json b/docs/openapi-rc.json index 24c8fc5..2c8a229 100644 --- a/docs/openapi-rc.json +++ b/docs/openapi-rc.json @@ -6296,6 +6296,449 @@ "409": { "description": "重复决定 9604 / 非法流转 9605", "content": { "*/*": { "schema": { "$ref": "#/components/schemas/ApiResponse" } } } } } } + }, + "/skills/word-cloud": { + "get": { + "tags": [ + "行业技能词云" + ], + "summary": "查询行业技能词云", + "description": "返回筛选范围内的全部技能，按 heat DESC, nameZh ASC, skillId ASC 排序，heat=null 最后；前端取前十条作为 TOP10。仅 STUDENT 可访问。", + "operationId": "getWordCloud", + "security": [ + { + "bearerAuth": [] + } + ], + "parameters": [ + { + "name": "industryId", + "in": "query", + "required": true, + "description": "行业 ID，必填正整数", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 1 + } + }, + { + "name": "occupationId", + "in": "query", + "required": true, + "description": "职位 ID，必填正整数；必须属于该行业", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 2 + } + }, + { + "name": "categoryId", + "in": "query", + "required": false, + "description": "技能分类 ID，可选正整数；省略时返回该职位下全部分类", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 101 + } + } + ], + "responses": { + "200": { + "description": "成功返回筛选范围内的完整技能列表", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponseSkillCloudResponse" + }, + "example": { + "code": 200, + "message": "ok", + "data": { + "industry": { + "id": 1, + "code": "INTERNET", + "name": "互联网" + }, + "occupation": { + "id": 1, + "code": "FRONTEND_DEVELOPER", + "name": "前端开发工程师" + }, + "category": null, + "total": 1, + "items": [ + { + "rank": 1, + "skillId": 26, + "skillCode": "REACT", + "nameZh": "React", + "nameEn": "React", + "sourceType": "O*NET", + "sourceCode": "15-1254.00|React|2.E.7.h", + "categoryId": 103, + "categoryName": "前端框架", + "heat": 4.38, + "heatSourceCode": "15-1254.00", + "heatElementId": "2.E.7.h", + "heatProxy": true + } + ] + }, + "requestId": "req-word-cloud", + "timestamp": "2026-09-15T10:00:00Z" + } + } + } + }, + "400": { + "description": "筛选参数缺失、非正整数或节点关系非法", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponse" + }, + "example": { + "code": 1001, + "message": "bad request", + "data": null, + "requestId": "req-1001", + "timestamp": "2026-09-15T10:03:00Z" + } + } + } + }, + "401": { + "description": "未登录或令牌无效，由全局 Security 入口返回裸 JSON", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/UnauthorizedResponse" + }, + "example": { + "code": 401, + "message": "not authenticated", + "timestamp": "" + } + } + } + }, + "403": { + "description": "已登录但不是 STUDENT 角色", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponse" + }, + "example": { + "code": 2002, + "message": "forbidden", + "data": null, + "requestId": "req-2002", + "timestamp": "2026-09-15T10:03:00Z" + } + } + } + } + } + } + }, + "/skills/match": { + "get": { + "tags": [ + "行业技能词云" + ], + "summary": "查询个人技能匹配度", + "description": "使用 JWT 对应学生的 skills_json，与当前筛选结果 TOP10 比对；matchScore=round(matchedCount*100/min(10,total))。仅 STUDENT 可访问。", + "operationId": "getSkillMatch", + "security": [ + { + "bearerAuth": [] + } + ], + "parameters": [ + { + "name": "industryId", + "in": "query", + "required": true, + "description": "行业 ID，必填正整数", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 1 + } + }, + { + "name": "occupationId", + "in": "query", + "required": true, + "description": "职位 ID，必填正整数；必须属于该行业", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 2 + } + }, + { + "name": "categoryId", + "in": "query", + "required": false, + "description": "技能分类 ID，可选正整数；省略时使用该职位下全部技能", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 101 + } + } + ], + "responses": { + "200": { + "description": "成功返回个人技能匹配结果", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponseSkillMatchResponse" + }, + "example": { + "code": 200, + "message": "ok", + "data": { + "matchScore": 50, + "matchedCount": 1, + "totalCount": 2, + "matchedSkills": [ + { + "rank": 1, + "skillId": 26, + "skillCode": "REACT", + "nameZh": "React", + "nameEn": "React", + "sourceType": "O*NET", + "sourceCode": "15-1254.00|React|2.E.7.h", + "categoryId": 103, + "categoryName": "前端框架", + "heat": 4.38, + "heatSourceCode": "15-1254.00", + "heatElementId": "2.E.7.h", + "heatProxy": true + } + ], + "missingSkills": [ + { + "rank": 2, + "skillId": 31, + "skillCode": "DOCKER", + "nameZh": "Docker", + "nameEn": "Docker", + "sourceType": "O*NET", + "sourceCode": "15-1299.08|Docker|2.E.7.h", + "categoryId": 105, + "categoryName": "工程化工具", + "heat": 4.12, + "heatSourceCode": "15-1299.08", + "heatElementId": "2.E.7.h", + "heatProxy": true + } + ] + }, + "requestId": "req-match", + "timestamp": "2026-09-15T10:01:00Z" + } + } + } + }, + "400": { + "description": "筛选参数缺失、非正整数或节点关系非法", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponse" + }, + "example": { + "code": 1001, + "message": "bad request", + "data": null, + "requestId": "req-1001", + "timestamp": "2026-09-15T10:03:00Z" + } + } + } + }, + "401": { + "description": "未登录或令牌无效，由全局 Security 入口返回裸 JSON", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/UnauthorizedResponse" + }, + "example": { + "code": 401, + "message": "not authenticated", + "timestamp": "" + } + } + } + }, + "403": { + "description": "已登录但不是 STUDENT 角色", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponse" + }, + "example": { + "code": 2002, + "message": "forbidden", + "data": null, + "requestId": "req-2002", + "timestamp": "2026-09-15T10:03:00Z" + } + } + } + } + } + } + }, + "/skills/learning-path": { + "get": { + "tags": [ + "行业技能词云" + ], + "summary": "生成个人学习路径", + "description": "只针对个人 TOP10 中未命中的技能，沿用热度顺序返回连续 priority 和确定性通用建议；不调用 AI，不返回课程链接、时长或进度。仅 STUDENT 可访问。", + "operationId": "getLearningPath", + "security": [ + { + "bearerAuth": [] + } + ], + "parameters": [ + { + "name": "industryId", + "in": "query", + "required": true, + "description": "行业 ID，必填正整数", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 1 + } + }, + { + "name": "occupationId", + "in": "query", + "required": true, + "description": "职位 ID，必填正整数；必须属于该行业", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 2 + } + }, + { + "name": "categoryId", + "in": "query", + "required": false, + "description": "技能分类 ID，可选正整数；省略时使用该职位下全部技能", + "schema": { + "type": "integer", + "format": "int64", + "minimum": 1, + "example": 101 + } + } + ], + "responses": { + "200": { + "description": "成功返回缺失技能的学习路径；没有缺口时列表为空", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponseSkillLearningPathResponse" + }, + "example": { + "code": 200, + "message": "ok", + "data": { + "total": 1, + "items": [ + { + "priority": 1, + "skillId": 26, + "skillName": "React", + "categoryId": 103, + "categoryName": "前端框架", + "suggestion": "先理解 React 的核心概念与生命周期，再通过一个小型项目练习集成。" + } + ] + }, + "requestId": "req-learning-path", + "timestamp": "2026-09-15T10:02:00Z" + } + } + } + }, + "400": { + "description": "筛选参数缺失、非正整数或节点关系非法", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponse" + }, + "example": { + "code": 1001, + "message": "bad request", + "data": null, + "requestId": "req-1001", + "timestamp": "2026-09-15T10:03:00Z" + } + } + } + }, + "401": { + "description": "未登录或令牌无效，由全局 Security 入口返回裸 JSON", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/UnauthorizedResponse" + }, + "example": { + "code": 401, + "message": "not authenticated", + "timestamp": "" + } + } + } + }, + "403": { + "description": "已登录但不是 STUDENT 角色", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ApiResponse" + }, + "example": { + "code": 2002, + "message": "forbidden", + "data": null, + "requestId": "req-2002", + "timestamp": "2026-09-15T10:03:00Z" + } + } + } + } + } + } } }, "components": { @@ -9679,6 +10122,285 @@ "requestId": { "type": "string" }, "timestamp": { "type": "string", "format": "date-time" } } + }, + "SkillCloudNode": { + "type": "object", + "properties": { + "id": { + "type": "integer", + "format": "int64" + }, + "code": { + "type": "string" + }, + "name": { + "type": "string" + } + } + }, + "SkillCloudItem": { + "type": "object", + "properties": { + "rank": { + "type": "integer", + "format": "int32", + "minimum": 1, + "description": "当前筛选范围内的稳定排名，从 1 开始。" + }, + "skillId": { + "type": "integer", + "format": "int64" + }, + "skillCode": { + "type": "string" + }, + "nameZh": { + "type": "string" + }, + "nameEn": { + "type": "string", + "nullable": true + }, + "sourceType": { + "type": "string", + "enum": [ + "ESCO", + "O*NET" + ], + "description": "技能身份主来源。" + }, + "sourceCode": { + "type": "string", + "description": "ESCO URI 或 O*NET 三元组标识。" + }, + "categoryId": { + "type": "integer", + "format": "int64" + }, + "categoryName": { + "type": "string" + }, + "heat": { + "type": "number", + "nullable": true, + "description": "O*NET Importance；无映射为 null，且排在非空热度之后。" + }, + "heatSourceCode": { + "type": "string", + "nullable": true, + "description": "热度来源 SOC。" + }, + "heatElementId": { + "type": "string", + "nullable": true, + "description": "热度来源 O*NET Element ID。" + }, + "heatProxy": { + "type": "boolean", + "description": "热度是否为代理映射。" + } + } + }, + "SkillCloudResponse": { + "type": "object", + "properties": { + "industry": { + "$ref": "#/components/schemas/SkillCloudNode" + }, + "occupation": { + "$ref": "#/components/schemas/SkillCloudNode" + }, + "category": { + "$ref": "#/components/schemas/SkillCloudNode", + "nullable": true + }, + "total": { + "type": "integer", + "format": "int32", + "minimum": 0 + }, + "items": { + "type": "array", + "items": { + "$ref": "#/components/schemas/SkillCloudItem" + } + } + } + }, + "SkillMatchResponse": { + "type": "object", + "properties": { + "matchScore": { + "type": "integer", + "format": "int32", + "minimum": 0, + "maximum": 100, + "description": "round(matchedCount*100/min(10,total))；total=0 时为 0。" + }, + "matchedCount": { + "type": "integer", + "format": "int32", + "minimum": 0 + }, + "totalCount": { + "type": "integer", + "format": "int32", + "minimum": 0, + "description": "实际匹配集合大小 min(10,total)。" + }, + "matchedSkills": { + "type": "array", + "items": { + "$ref": "#/components/schemas/SkillCloudItem" + } + }, + "missingSkills": { + "type": "array", + "items": { + "$ref": "#/components/schemas/SkillCloudItem" + } + } + } + }, + "SkillLearningPathItem": { + "type": "object", + "properties": { + "priority": { + "type": "integer", + "format": "int32", + "minimum": 1 + }, + "skillId": { + "type": "integer", + "format": "int64" + }, + "skillName": { + "type": "string" + }, + "categoryId": { + "type": "integer", + "format": "int64" + }, + "categoryName": { + "type": "string" + }, + "suggestion": { + "type": "string", + "description": "按技能分类生成的通用建议，不含课程、AI、链接、时长或进度。" + } + } + }, + "SkillLearningPathResponse": { + "type": "object", + "properties": { + "total": { + "type": "integer", + "format": "int32", + "minimum": 0 + }, + "items": { + "type": "array", + "items": { + "$ref": "#/components/schemas/SkillLearningPathItem" + } + } + } + }, + "ApiResponseSkillCloudResponse": { + "type": "object", + "properties": { + "code": { + "type": "integer", + "format": "int32" + }, + "message": { + "type": "string" + }, + "data": { + "$ref": "#/components/schemas/SkillCloudResponse" + }, + "requestId": { + "type": "string" + }, + "timestamp": { + "type": "string", + "format": "date-time" + } + } + }, + "ApiResponseSkillMatchResponse": { + "type": "object", + "properties": { + "code": { + "type": "integer", + "format": "int32" + }, + "message": { + "type": "string" + }, + "data": { + "$ref": "#/components/schemas/SkillMatchResponse" + }, + "requestId": { + "type": "string" + }, + "timestamp": { + "type": "string", + "format": "date-time" + } + } + }, + "ApiResponseSkillLearningPathResponse": { + "type": "object", + "properties": { + "code": { + "type": "integer", + "format": "int32" + }, + "message": { + "type": "string" + }, + "data": { + "$ref": "#/components/schemas/SkillLearningPathResponse" + }, + "requestId": { + "type": "string" + }, + "timestamp": { + "type": "string", + "format": "date-time" + } + } + }, + "UnauthorizedResponse": { + "type": "object", + "additionalProperties": false, + "required": [ + "code", + "message", + "timestamp" + ], + "properties": { + "code": { + "type": "integer", + "format": "int32", + "enum": [ + 401 + ] + }, + "message": { + "type": "string", + "enum": [ + "not authenticated" + ] + }, + "timestamp": { + "type": "string", + "enum": [ + "" + ] + } + } } }, "securitySchemes": { diff --git a/docs/skill-cloud-api.md b/docs/skill-cloud-api.md new file mode 100644 index 0000000..476de9d --- /dev/null +++ b/docs/skill-cloud-api.md @@ -0,0 +1,275 @@ +# 行业技能词云、个人匹配度与学习路径 API + +> 版本：N03 设计锁定版 +> 状态：已编码实现 +> 适用范围：学生端只读接口。本文描述 `GET /api/v1/skills/word-cloud`、`GET /api/v1/skills/match`、`GET /api/v1/skills/learning-path`。 +> Base URL：`http://127.0.0.1:8080/api/v1`（生产环境替换域名，路径不变）。 + +## 1. 权限与共同参数 + +三个接口均： + +- 仅允许 `STUDENT` 访问。 +- 必须携带 `Authorization: Bearer `，身份只从 JWT/SecurityContext 获取，不接受 `userId`、`studentId` 等身份注入参数。 +- 成功、`400` 和 `403` 响应使用统一 `ApiResponse`；匿名或无效 JWT 的 `401` 由全局 Security 入口直接返回裸 JSON，不包含 `data` 和 `requestId`。 +- 只读，不写入学习进度、学生资料或词库数据。 + +查询参数： + +| 参数 | 类型 | 必填 | 说明 | +|---|---|---|---| +| `industryId` | long | 是 | 词库行业 ID，必须为正整数，且必须是真实存在的行业。 | +| `occupationId` | long | 是 | 词库职位 ID，必须为正整数，且必须属于该行业。 | +| `categoryId` | long | 否 | 技能分类 ID，必须为正整数；省略时使用该职位下全部技能分类。传入时必须属于目标职位。 | + +任一必填参数缺失、非正整数，或行业—职位—分类关系非法时返回 `400`。如果未登录返回 `401`，非 STUDENT 用户返回 `403`。 + +## 2. 通用响应与排序 + +三个接口的 `data` 都使用 camelCase 字段。词云和 TOP10 使用同一份服务端排序结果： + +1. `heat` 从高到低；`heat` 为 `null` 的技能排在所有非空热度之后。 +2. 热度相同时按 `nameZh` 升序。 +3. 其余情况按 `skillId` 升序，保证重复请求顺序稳定。 +4. `rank` 从 1 连续递增。 + +`heat` 来自 O*NET Importance。无可靠 O*NET Importance 映射的 ESCO 技能仍返回，但 `heat` 为 `null`。TOP10 不在后端单独存储或单独排序；前端直接取词云 `data.items` 的前 `min(10, data.total)` 项。 + +技能身份来源： + +- `sourceType`：`ESCO` 或 `O*NET`，表示技能条目的主来源。 +- `sourceCode`：可追溯来源标识。ESCO 为技能 URI；O*NET 补充技能为 `SOC|Workplace Example|Element ID`。 +- `heatSourceCode`：热度来源 SOC；没有 O*NET 热度时为 `null`。 +- `heatElementId`：O*NET Importance 的来源 Element ID；没有时为 `null`。 +- `heatProxy`：该条热度是否为代理映射。代理值不改变排序规则，但前端可据此展示来源说明。 + +## 3. 查询词云 + +```http +GET /api/v1/skills/word-cloud?industryId=1&occupationId=2&categoryId=101 +Authorization: Bearer +``` + +`categoryId` 可省略： + +```http +GET /api/v1/skills/word-cloud?industryId=1&occupationId=2 +``` + +成功响应为 `ApiResponse`。`SkillCloudResponse` 返回筛选节点、总数和当前范围内的完整技能列表： + +```json +{ + "code": 200, + "message": "ok", + "data": { + "industry": { "id": 1, "code": "INTERNET", "name": "互联网" }, + "occupation": { "id": 1, "code": "FRONTEND_DEVELOPER", "name": "前端开发工程师" }, + "category": { "id": 103, "code": "FRONTEND_FRAMEWORK", "name": "前端框架" }, + "total": 1, + "items": [ + { + "rank": 1, + "skillId": 26, + "skillCode": "REACT", + "nameZh": "React", + "nameEn": "React", + "sourceType": "O*NET", + "sourceCode": "15-1254.00|React|2.E.7.h", + "categoryId": 103, + "categoryName": "前端框架", + "heat": 4.38, + "heatSourceCode": "15-1254.00", + "heatElementId": "2.E.7.h", + "heatProxy": true + } + ] + }, + "requestId": "req-123", + "timestamp": "2026-09-15T10:00:00Z" +} +``` + +`category` 在未传 `categoryId` 时为 `null`。合法范围但暂时没有技能时返回成功、`total=0`、`items=[]`。前端根据 `items` 渲染词云，并截取前十条作为 TOP10。 + +## 4. 查询个人技能匹配度 + +```http +GET /api/v1/skills/match?industryId=1&occupationId=2&categoryId=101 +Authorization: Bearer +``` + +匹配集合严格取当前筛选结果的 TOP10。系统从当前 JWT 学生的 `student_profiles.skills_json` 读取技能，使用标准技能名、英文名、技能代码和别名归一化匹配。大小为 `total` 的筛选范围使用 `min(10, total)` 作为分母： + +```text +matchScore = round(matchedCount * 100 / min(10, total)) +``` + +- `total=0` 时 `matchScore=0`，命中列表和缺失列表为空。 +- 范围少于 10 项时按实际技能数计算。 +- 重复技能或指向同一标准技能的多个别名只计一次。 +- 示例：范围 15 条、命中 5 条，`matchScore=33`。 +- 原始 `skills_json`、手机号、邮箱和简历内容不会出现在响应中。 + +成功响应为 `ApiResponse`： + +```json +{ + "code": 200, + "message": "ok", + "data": { + "matchScore": 50, + "matchedCount": 1, + "totalCount": 2, + "matchedSkills": [ + { + "rank": 1, + "skillId": 26, + "skillCode": "REACT", + "nameZh": "React", + "nameEn": "React", + "sourceType": "O*NET", + "sourceCode": "15-1254.00|React|2.E.7.h", + "categoryId": 103, + "categoryName": "前端框架", + "heat": 4.38, + "heatSourceCode": "15-1254.00", + "heatElementId": "2.E.7.h", + "heatProxy": true + } + ], + "missingSkills": [ + { + "rank": 2, + "skillId": 31, + "skillCode": "DOCKER", + "nameZh": "Docker", + "nameEn": "Docker", + "sourceType": "O*NET", + "sourceCode": "15-1299.08|Docker|2.E.7.h", + "categoryId": 105, + "categoryName": "工程化工具", + "heat": 4.12, + "heatSourceCode": "15-1299.08", + "heatElementId": "2.E.7.h", + "heatProxy": true + } + ] + }, + "requestId": "req-456", + "timestamp": "2026-09-15T10:01:00Z" +} +``` + +`matchedSkills` 与 `missingSkills` 的并集等于当前 TOP10，两个列表各自保持词云排序；`totalCount` 是实际匹配集合大小。 + +## 5. 生成学习路径 + +```http +GET /api/v1/skills/learning-path?industryId=1&occupationId=2&categoryId=101 +Authorization: Bearer +``` + +学习路径只消费个人匹配结果中的 `missingSkills`： + +- 只包含当前 TOP10 中未命中的技能，不包含已命中技能或 TOP10 之外的技能。 +- 顺序与 `missingSkills` 完全一致，`priority` 从 1 连续递增。 +- 每项只返回技能 ID、名称、分类、优先级和简化学习建议。 +- 后端不调用 AI、课程平台或外部知识库；不会返回课程链接、讲师、价格、学习时长或完成度。 +- 系统不持久化学习进度或完成状态。 +- 全部命中或合法范围没有技能时返回 `total=0`、`items=[]`。 + +建议模板按技能分类确定性生成： + +| 模板类别 | 建议方向 | +|---|---| +| 编程语言 | 基础语法、常见数据结构、至少一个编码练习。 | +| 框架 | 核心概念与生命周期、小型项目集成实践。 | +| 工具 | 安装与基础配置、结合工程完成实际操作。 | +| 通用 | 核心概念和常见应用场景、简短实践巩固。 | + +```json +{ + "code": 200, + "message": "ok", + "data": { + "total": 1, + "items": [ + { + "priority": 1, + "skillId": 26, + "skillName": "React", + "categoryId": 103, + "categoryName": "前端框架", + "suggestion": "先理解 React 的核心概念与生命周期，再通过一个小型项目练习集成。" + } + ] + }, + "requestId": "req-789", + "timestamp": "2026-09-15T10:02:00Z" +} +``` + +## 6. 错误响应示例 + +`400` 和 `403` 使用统一 `ApiResponse`；`401` 由全局 Security 入口返回裸 JSON。以下示例与运行时契约一致。 + +```json +{ + "code": 1001, + "message": "bad request", + "data": null, + "requestId": "req-400", + "timestamp": "2026-09-15T10:03:00Z" +} +``` + +| HTTP | 场景 | 典型 `code` | +|---|---|---| +| `400` | `industryId`/`occupationId` 缺失、非正整数，或节点关系非法 | `1001` | +| `401` | 未携带有效 JWT 或令牌无效/过期 | `401` | +| `403` | 已登录但不是 STUDENT | `2002` | + +示例 401： + +```json +{ + "code": 401, + "message": "not authenticated", + "timestamp": "" +} +``` + +`401` 响应不包含 `data` 和 `requestId`。 + +示例 403： + +```json +{ + "code": 2002, + "message": "forbidden", + "data": null, + "requestId": "req-403", + "timestamp": "2026-09-15T10:05:00Z" +} +``` + +## 7. 字段参考 + +### SkillCloudNode + +`id`、`code`、`name`。`category` 节点在未指定分类时为 `null`。 + +### SkillCloudItem + +`rank`、`skillId`、`skillCode`、`nameZh`、`nameEn`、`sourceType`、`sourceCode`、`categoryId`、`categoryName`、`heat`、`heatSourceCode`、`heatElementId`、`heatProxy`。 + +### SkillMatchResponse + +`matchScore`、`matchedCount`、`totalCount`、`matchedSkills`、`missingSkills`。 + +### SkillLearningPathItem / SkillLearningPathResponse + +`SkillLearningPathItem`：`priority`、`skillId`、`skillName`、`categoryId`、`categoryName`、`suggestion`。 + +`SkillLearningPathResponse`：`total`、`items`。 diff --git a/docs/skill-taxonomy-sources.md b/docs/skill-taxonomy-sources.md new file mode 100644 index 0000000..049e69a --- /dev/null +++ b/docs/skill-taxonomy-sources.md @@ -0,0 +1,79 @@ +# 行业技能词库来源与覆盖说明 + +本说明是仓库面向实现和评审的技能词库来源基线，内容与 `changes/n03-skill-cloud/data/source-manifest.md` 保持同一事实集合。 + +## 1. 固定来源 + +| 来源 | 固定版本 | 用途 | 许可/复用条件 | 官方地址与归属 | +| --- | --- | --- | --- | --- | +| ESCO | ESCO dataset v1.2.1（Last update 10/12/2025，发布页记录 Release v5.7.0） | 提供行业/职位映射所需的职业—技能—技能分类主关系、稳定来源 ID 和层级结构 | 依据 Commission Decision 2011/833/EU，可免费下载、使用、复制和复用。文档和产品界面须保留官方署名语句：`This service uses the ESCO classification of the European Commission.` | 数据集下载页：https://esco.ec.europa.eu/en/use-esco/download
复用条件说明：https://esco.ec.europa.eu/en/about-esco/faq | +| O*NET | O*NET 31.0 Database | 为 ESCO 未收录的具体技术提供 `software_skills` 补充来源，并通过职业/工作活动 Importance 提供首版词云热度 `heat` | License 为 CC BY 4.0，归属与许可说明见 https://creativecommons.org/licenses/by/4.0/ | 官方数据库页：https://www.onetcenter.org/database.html | + +运行时不得访问 ESCO、O*NET 或任何在线词库服务。所有首版数据必须随 Flyway 迁移进入本地数据库，并保留来源版本与归属信息。 + +## 2. 首版覆盖边界 + +首版只覆盖互联网、金融、教育三类行业及下列 10 个已列代表性职位。该矩阵定义导入上限和最小覆盖范围，不要求在全量上游数据中导入未列出的职位。 + +| 行业 | 代表性职位 | 首批技能分类范围 | +| --- | --- | --- | +| 互联网 | 前端开发工程师 | 编程语言、Web 基础、前端框架、工程化工具、浏览器与调试、测试、版本控制与协作 | +| 互联网 | 后端开发工程师 | 编程语言、后端框架与平台、数据库、缓存与消息、云/DevOps、测试、工程化工具、安全 | +| 互联网 | 测试工程师 | 编程语言、测试设计、自动化测试框架、接口与数据库测试、性能测试、CI/CD 与工程化工具 | +| 互联网 | DevOps 工程师 | 编程语言与脚本、云平台、容器与编排、CI/CD、配置管理与 IaC、可观测性、安全、数据库 | +| 金融 | 金融数据分析师 | 编程语言、数据分析与可视化、数据库与数据仓库、统计与建模、金融计算框架/平台、金融领域知识、工程化工具 | +| 金融 | 风控建模工程师 | 编程语言、统计与建模、风险模型、建模框架/平台、数据处理与数据库、模型验证、部署与工程化工具 | +| 金融 | 量化开发工程师 | 编程语言、量化建模框架/平台、市场与金融数据、数据库、低延迟与高性能工程、测试、云/DevOps | +| 教育 | 教育产品经理 | 教育领域知识、产品分析方法、需求与流程、数据分析、教育平台与工具、协作与工程化工具 | +| 教育 | 教学设计师 | 教学设计、课程与学习平台、数字化内容工具、评估设计、学习数据分析、无障碍与规范 | +| 教育 | 教育数据分析师 | 编程语言、数据分析与可视化、数据库与 SQL、统计与建模、教育领域知识、报表与工程化工具 | +计数说明：任务给出的逐项职位清单为互联网 4 个、金融 3 个、教育 3 个，共 10 个。验收文字中的“11 个职位”与该枚举不一致；本清单严格覆盖逐项列出的 10 个职位，不自行新增未批准职位。 + +矩阵中的分类是每个职位首批应覆盖的能力域；T3 只导入这些分类下已确定稳定来源 ID、标准技能和中文标准名的技能，不在此文档中展开具体 `INSERT` 行。 + +## 3. 导入规则 + +1. **职责分工**：T2 只建立七张词库表、外键、唯一约束和查询索引；T3 只导入上述三类行业、10 个已列代表性职位及其关联技能分类、技能、别名和 O*NET Importance。 +2. **代表性子集与混合来源**：不导入 ESCO/O*NET 全量数据；ESCO 是职位、通用技能和分类主表；ESCO 无可靠节点的具体技术使用 O*NET `software_skills` 补充；不把平台自由文本岗位或行业直接当作标准词库节点。 +3. **来源 ID 不变**：ESCO/O*NET 来源 ID 原样保存在稳定来源字段中；项目不得重编号、覆盖或伪造上游 ID。 +4. **中文标准名必填**：每个入选技能必须有非空 `name_zh` 中文标准名；中文名称由项目维护，但不替代或修改上游来源 ID。 +5. **热度来源**：每条非空 Importance 必须保存 O*NET SOC、Element ID 和代理标记；技能无法映射 O*NET Importance 时必须保留，写入 `importance=null`；查询时映射为 `heat=null`，并排在有热度技能之后。 +6. **项目映射责任**：项目维护中文名称、别名、技能分类和行业/职位映射，不修改开源来源 ID。行业/职位映射、细粒度分类和跨源技能对应关系均作为项目适配层记录。 +7. **别名归一化**：生成 `normalized_alias` 时依次执行 Unicode NFKC、首尾空白移除、大小写折叠、连字符转空格、连续空白折叠为单空格、句点移除；`+` 和 `#` 必须保留，不能删除或替换。 +8. **来源可追溯**：来源清单写入 `skill_source_versions`，至少记录来源名称、固定版本、许可、归属 URL 和导入时间；O*NET 补充技术的 `source_code` 使用真实 `SOC|Workplace Example|Element ID` 组合，职位技能关系记录 O*NET SOC、Element ID、代理标记及关联的来源版本。 +9. **迁移不可改写**：T2/T3 计划使用 `V35__skill_cloud_taxonomy.sql` 和 `V36__skill_cloud_seed.sql`。执行前若编号已被占用，必须回退规划并刷新编号；后续词库更新只能新增 Flyway 迁移，不得改写 V1–V34 或任何已执行迁移。 +10. **运行时离线**：接口运行时不得调用外部词库、AI、课程平台或同步服务；排名只读取本地 Flyway 数据。 + +## 4. 来源版本、许可与归属 URL + +| 数据落点 | 来源名称 | 版本字段固定值 | 许可字段固定值 | 归属 URL | +| --- | --- | --- | --- | --- | +| 职业、技能、技能分类及关系 | ESCO | `1.2.1 (Release v5.7.0, Last update 10/12/2025)` | `Commission Decision 2011/833/EU` | https://esco.ec.europa.eu/en/use-esco/download | +| 职位技能 Importance | O*NET | `31.0 Database` | `CC BY 4.0` | https://www.onetcenter.org/database.html | + +ESCO 产品署名必须使用：`This service uses the ESCO classification of the European Commission.` + +## 5. 映射责任边界 + +| 事项 | 上游来源责任 | 项目责任 | +| --- | --- | --- | +| ESCO 职业、技能、分类层级和来源 ID | 固定且可追溯 | 原样保存，不修改 ID | +| O*NET Importance 数值 | 固定且可追溯 | 原样保存；无映射时写 `null`，不造权重 | +| 中文名称和别名 | 不保证完整中文覆盖 | 补齐中文标准名、别名及归一化值 | +| 细粒度技能分类 | 不保证符合本项目展示粒度 | 在职位下维护分类和展示名称 | +| 行业/职位映射 | 不保证覆盖三类项目行业和中文职位名 | 维护行业—职位—分类—技能映射 | +| 跨源技能与重要性关联 | 不提供项目所需的稳定本地关系 | 维护 ESCO 技能到 O*NET `software_skills` 的补充映射，以及每条 Importance 的 SOC/Element ID/代理标记 | + +## 6. 验收检查项 + +- AC-1：`changes/n03-skill-cloud/data/source-manifest.md` 与 `docs/skill-taxonomy-sources.md` 均存在且非空，来源版本、许可、归属 URL、覆盖矩阵和导入规则一致。 +- AC-2：文档明确覆盖互联网、金融、教育三类行业及全部 10 个已列代表性职位，并为每个职位给出首批技能分类范围。 +- AC-3：文档明确固定 ESCO dataset v1.2.1（Last update 10/12/2025，发布页记录 Release v5.7.0）和 O*NET 31.0 Database。 +- AC-4：文档明确 ESCO 依据 Commission Decision 2011/833/EU 可免费下载、使用、复制和复用，并原样保留指定署名语句。 +- AC-5：文档明确 O*NET License 为 CC BY 4.0，并记录官方数据库页与许可 URL。 +- AC-6：文档明确项目维护中文名称、别名、技能分类和行业/职位映射，不修改开源来源 ID。 +- AC-7：文档明确 T2 建结构、T3 导数据，只导入代表性职位和关联技能；技能中文标准名必填。 +- AC-8：文档明确 O*NET 补充技术、每条 Importance 的 SOC/Element ID/代理标记，以及无映射时保留技能并写入 `importance=null`、查询侧映射为 `heat=null`。 +- AC-9：文档明确别名归一化遵循 NFKC、首尾空白、大小写、连字符、连续空白、句点规则，且保留 `+`、`#`。 +- AC-10：文档明确后续词库更新只能新增 Flyway 迁移，不得改写 V1–V34 或已执行迁移。 +- AC-11：来源、覆盖、导入与责任边界中不得出现未决占位或延期确认语句。 \ No newline at end of file diff --git a/docs/verify_openapi.py b/docs/verify_openapi.py index 7470085..6f3b5ca 100644 --- a/docs/verify_openapi.py +++ b/docs/verify_openapi.py @@ -6,9 +6,10 @@ p = PROJECT_ROOT / 'docs' / 'openapi-rc.json' d = json.load(open(p, encoding='utf-8')) print('paths:', len(d['paths'])) -assert len(d['paths']) == 126, f"Expected 126 paths, got {len(d['paths'])}" +assert len(d['paths']) == 129, f"Expected 129 paths, got {len(d['paths'])}" paths = d['paths'] + required = [ '/review-weight-templates', '/profile/me/review-weights', @@ -66,6 +67,9 @@ required = [ '/admin/reports', '/admin/reports/{id}', '/admin/reports/{id}/decision', + '/skills/word-cloud', + '/skills/match', + '/skills/learning-path', ] for path in required: assert path in paths, f"Missing path: {path}" @@ -135,6 +139,68 @@ assert 'get' in paths['/admin/reports'], "Missing GET /admin/reports" assert 'get' in paths['/admin/reports/{id}'], "Missing GET /admin/reports/{id}" assert 'post' in paths['/admin/reports/{id}/decision'], "Missing POST /admin/reports/{id}/decision" print(" V33 report/block endpoints present (9 operations / 8 paths)") + +# Skill cloud endpoints (3 paths / 3 GET operations) +for skill_path in ('/skills/word-cloud', '/skills/match', '/skills/learning-path'): + assert 'get' in paths[skill_path], f"Missing GET {skill_path}" +skill_operation_ids = { + '/skills/word-cloud': 'getWordCloud', + '/skills/match': 'getSkillMatch', + '/skills/learning-path': 'getLearningPath', +} +for skill_path, skill_operation_id in skill_operation_ids.items(): + assert paths[skill_path]['get'].get('operationId') == skill_operation_id + http_methods = {'get', 'post', 'put', 'patch', 'delete', 'options', 'head', 'trace'} + matching_operations = [ + (method, path) + for path, path_item in paths.items() + for method, operation in path_item.items() + if method in http_methods + and operation.get('operationId') == skill_operation_id + ] + assert matching_operations == [('get', skill_path)], ( + f'{skill_operation_id} must not conflict: expected GET {skill_path}, got {matching_operations}' + ) +assert paths['/skills/word-cloud']['get']['responses']['200']['content']['application/json']['schema']['$ref'] == '#/components/schemas/ApiResponseSkillCloudResponse' +assert paths['/skills/match']['get']['responses']['200']['content']['application/json']['schema']['$ref'] == '#/components/schemas/ApiResponseSkillMatchResponse' +assert paths['/skills/learning-path']['get']['responses']['200']['content']['application/json']['schema']['$ref'] == '#/components/schemas/ApiResponseSkillLearningPathResponse' + +for skill_path in ('/skills/word-cloud', '/skills/match', '/skills/learning-path'): + unauthorized = paths[skill_path]['get']['responses']['401'] + assert unauthorized['description'] == '未登录或令牌无效，由全局 Security 入口返回裸 JSON' + assert unauthorized['content']['application/json']['schema']['$ref'] == '#/components/schemas/UnauthorizedResponse' + example = unauthorized['content']['application/json']['example'] + assert example == {'code': 401, 'message': 'not authenticated', 'timestamp': ''} + assert 'requestId' not in example + +schemas = d['components']['schemas'] +unauthorized_schema = schemas.get('UnauthorizedResponse') +assert unauthorized_schema is not None, 'Missing schema: UnauthorizedResponse' +assert unauthorized_schema.get('additionalProperties') is False +assert set(unauthorized_schema.get('required', [])) == {'code', 'message', 'timestamp'} +assert set(unauthorized_schema.get('properties', {})) == {'code', 'message', 'timestamp'} +skill_schema_fields = { + 'SkillCloudNode': {'id', 'code', 'name'}, + 'SkillCloudItem': {'rank', 'skillId', 'skillCode', 'nameZh', 'nameEn', 'sourceType', 'sourceCode', 'categoryId', 'categoryName', 'heat', 'heatSourceCode', 'heatElementId', 'heatProxy'}, + 'SkillCloudResponse': {'industry', 'occupation', 'category', 'total', 'items'}, + 'SkillMatchResponse': {'matchScore', 'matchedCount', 'totalCount', 'matchedSkills', 'missingSkills'}, + 'SkillLearningPathItem': {'priority', 'skillId', 'skillName', 'categoryId', 'categoryName', 'suggestion'}, + 'SkillLearningPathResponse': {'total', 'items'}, +} +for schema_name, expected_fields in skill_schema_fields.items(): + schema = schemas.get(schema_name) + assert schema is not None, f'Missing schema: {schema_name}' + assert expected_fields <= set(schema.get('properties', {})), f'Schema fields missing: {schema_name}' + +word_cloud_example = paths['/skills/word-cloud']['get']['responses']['200']['content']['application/json']['example']['data'] +assert word_cloud_example['total'] == len(word_cloud_example['items']) +match_example = paths['/skills/match']['get']['responses']['200']['content']['application/json']['example']['data'] +assert match_example['matchedCount'] == len(match_example['matchedSkills']) +assert match_example['totalCount'] == len(match_example['matchedSkills']) + len(match_example['missingSkills']) +assert match_example['matchScore'] == round(match_example['matchedCount'] * 100 / match_example['totalCount']) +learning_path_example = paths['/skills/learning-path']['get']['responses']['200']['content']['application/json']['example']['data'] +assert learning_path_example['total'] == len(learning_path_example['items']) +print(" Skill cloud endpoints and schemas present (3 GET operations)") print(" Offer advance endpoints all present") print(" 26 operations all present") print(" Account deletion endpoints present (3 operations)") diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/config/SecurityConfig.java b/mydreamjob-server/src/main/java/com/mydreamjob/config/SecurityConfig.java index fc11d84..0394c90 100644 --- a/mydreamjob-server/src/main/java/com/mydreamjob/config/SecurityConfig.java +++ b/mydreamjob-server/src/main/java/com/mydreamjob/config/SecurityConfig.java @@ -74,6 +74,8 @@ public class SecurityConfig { .requestMatchers("/auth/register").permitAll() // 当前用户 .requestMatchers("/auth/me").authenticated() + // 行业技能词云（需登录，仅 STUDENT，Controller 层校验角色） + .requestMatchers(HttpMethod.GET, "/skills/**").authenticated() // 企业资料（需登录） .requestMatchers("/companies/me/**").authenticated() // 企业公开岗位列表（匿名 GET） diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudController.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudController.java new file mode 100644 index 0000000..01aa843 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudController.java @@ -0,0 +1,207 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.common.api.ApiResponse; +import com.mydreamjob.common.exception.BusinessException; +import com.mydreamjob.common.exception.ErrorCode; +import com.mydreamjob.profile.ProfileService; +import com.mydreamjob.skill.dto.SkillCloudResponse; +import com.mydreamjob.skill.dto.SkillLearningPathResponse; +import com.mydreamjob.skill.dto.SkillMatchResponse; +import io.swagger.v3.oas.annotations.Operation; +import io.swagger.v3.oas.annotations.Parameter; +import io.swagger.v3.oas.annotations.media.Content; +import io.swagger.v3.oas.annotations.media.ExampleObject; +import io.swagger.v3.oas.annotations.media.Schema; +import io.swagger.v3.oas.annotations.responses.ApiResponses; +import io.swagger.v3.oas.annotations.security.SecurityRequirement; +import io.swagger.v3.oas.annotations.tags.Tag; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.validation.constraints.Positive; +import lombok.RequiredArgsConstructor; +import org.springframework.http.ResponseEntity; +import org.springframework.security.core.Authentication; +import org.springframework.security.core.context.SecurityContextHolder; +import org.springframework.validation.annotation.Validated; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RequestParam; +import org.springframework.web.bind.annotation.RestController; + +/** + * 行业技能词云、个人匹配度与学习路径接口。 + */ +@Tag(name = "行业技能词云", description = "按行业、职位和技能分类查询热门技能、个人匹配度与学习路径") +@RestController +@Validated +@RequiredArgsConstructor +public class SkillCloudController { + + private final SkillCloudQueryService skillCloudQueryService; + private final SkillMatchService skillMatchService; + private final SkillLearningPathService skillLearningPathService; + private final ProfileService profileService; + + @Operation( + operationId = "getWordCloud", + summary = "查询行业技能词云", + description = "返回当前筛选范围内的全部技能，按热度降序排列；前端取前十条作为 TOP10。仅 STUDENT 可访问。" + ) + @SecurityRequirement(name = "bearerAuth") + @ApiResponses({ + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "200", + description = "成功返回筛选范围内的全部技能", + content = @Content( + mediaType = "application/json", + examples = @ExampleObject( + name = "success", + value = "{\"code\":200,\"message\":\"ok\",\"data\":{\"industry\":{\"id\":1,\"code\":\"INTERNET\",\"name\":\"互联网\"},\"occupation\":{\"id\":1,\"code\":\"FRONTEND_DEVELOPER\",\"name\":\"前端开发工程师\"},\"category\":null,\"total\":1,\"items\":[{\"rank\":1,\"skillId\":26,\"skillCode\":\"REACT\",\"nameZh\":\"React\",\"nameEn\":\"React\",\"sourceType\":\"O*NET\",\"sourceCode\":\"15-1254.00|React|2.E.7.h\",\"categoryId\":103,\"categoryName\":\"前端框架\",\"heat\":4.38,\"heatSourceCode\":\"15-1254.00\",\"heatElementId\":\"2.E.7.h\",\"heatProxy\":true}]}}" + ) + ) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "400", description = "筛选参数缺失、非正整数或节点关系非法", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "401", description = "未登录", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "403", description = "已登录但不是 STUDENT 角色", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ) + }) + @GetMapping("/skills/word-cloud") + public ResponseEntity> getWordCloud( + @Parameter(description = "行业 ID，必填正整数", required = true, example = "1") + @Positive(message = "industryId must be positive") + @RequestParam Long industryId, + @Parameter(description = "职位 ID，必填正整数", required = true, example = "1") + @Positive(message = "occupationId must be positive") + @RequestParam Long occupationId, + @Parameter(description = "技能分类 ID，可选正整数；省略时返回职位下全部分类", required = false, example = "101") + @Positive(message = "categoryId must be positive") + @RequestParam(required = false) Long categoryId, + HttpServletRequest request) { + Long userId = resolveAuthenticatedStudent(); + SkillCloudResponse data = skillCloudQueryService.query(industryId, occupationId, categoryId); + return ResponseEntity.ok(ApiResponse.ok(data, getRequestId(request))); + } + + @Operation( + operationId = "getSkillMatch", + summary = "查询个人技能匹配度", + description = "使用当前 STUDENT 的个人技能，与筛选结果热度 TOP10 比对并返回命中、缺失和匹配度。仅 STUDENT 可访问。" + ) + @SecurityRequirement(name = "bearerAuth") + @ApiResponses({ + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "200", + description = "成功返回个人技能匹配结果", + content = @Content( + mediaType = "application/json", + examples = @ExampleObject( + name = "success", + value = "{\"code\":200,\"message\":\"ok\",\"data\":{\"matchScore\":50,\"matchedCount\":1,\"totalCount\":2,\"matchedSkills\":[{\"rank\":1,\"skillId\":26,\"skillCode\":\"REACT\",\"nameZh\":\"React\",\"nameEn\":\"React\",\"sourceType\":\"O*NET\",\"sourceCode\":\"15-1254.00|React|2.E.7.h\",\"categoryId\":103,\"categoryName\":\"前端框架\",\"heat\":4.38,\"heatSourceCode\":\"15-1254.00\",\"heatElementId\":\"2.E.7.h\",\"heatProxy\":true}],\"missingSkills\":[{\"rank\":2,\"skillId\":31,\"skillCode\":\"DOCKER\",\"nameZh\":\"Docker\",\"nameEn\":\"Docker\",\"sourceType\":\"O*NET\",\"sourceCode\":\"15-1299.08|Docker|2.E.7.h\",\"categoryId\":105,\"categoryName\":\"工程化工具\",\"heat\":4.12,\"heatSourceCode\":\"15-1299.08\",\"heatElementId\":\"2.E.7.h\",\"heatProxy\":true}]}}" + ) + ) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "400", description = "筛选参数缺失、非正整数或节点关系非法", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "401", description = "未登录", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "403", description = "已登录但不是 STUDENT 角色", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ) + }) + @GetMapping("/skills/match") + public ResponseEntity> getMatch( + @Parameter(description = "行业 ID，必填正整数", required = true, example = "1") + @Positive(message = "industryId must be positive") + @RequestParam Long industryId, + @Parameter(description = "职位 ID，必填正整数", required = true, example = "1") + @Positive(message = "occupationId must be positive") + @RequestParam Long occupationId, + @Parameter(description = "技能分类 ID，可选正整数；省略时使用职位下全部技能", required = false, example = "101") + @Positive(message = "categoryId must be positive") + @RequestParam(required = false) Long categoryId, + HttpServletRequest request) { + Long userId = resolveAuthenticatedStudent(); + SkillMatchResponse data = skillMatchService.match(userId, industryId, occupationId, categoryId); + return ResponseEntity.ok(ApiResponse.ok(data, getRequestId(request))); + } + + @Operation( + operationId = "getLearningPath", + summary = "生成个人学习路径", + description = "仅针对个人 TOP10 中未命中的技能，按热度顺序返回优先级和通用学习建议。仅 STUDENT 可访问。" + ) + @SecurityRequirement(name = "bearerAuth") + @ApiResponses({ + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "200", + description = "成功返回缺失技能的学习路径；没有缺口时列表为空", + content = @Content( + mediaType = "application/json", + examples = @ExampleObject( + name = "success", + value = "{\"code\":200,\"message\":\"ok\",\"data\":{\"total\":1,\"items\":[{\"priority\":1,\"skillId\":26,\"skillName\":\"React\",\"categoryId\":103,\"categoryName\":\"前端框架\",\"suggestion\":\"先理解 React 的核心概念与生命周期，再通过一个小型项目练习集成。\"}]}}" + ) + ) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "400", description = "筛选参数缺失、非正整数或节点关系非法", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "401", description = "未登录", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ), + @io.swagger.v3.oas.annotations.responses.ApiResponse( + responseCode = "403", description = "已登录但不是 STUDENT 角色", + content = @Content(schema = @Schema(implementation = ApiResponse.class)) + ) + }) + @GetMapping("/skills/learning-path") + public ResponseEntity> getLearningPath( + @Parameter(description = "行业 ID，必填正整数", required = true, example = "1") + @Positive(message = "industryId must be positive") + @RequestParam Long industryId, + @Parameter(description = "职位 ID，必填正整数", required = true, example = "1") + @Positive(message = "occupationId must be positive") + @RequestParam Long occupationId, + @Parameter(description = "技能分类 ID，可选正整数；省略时使用职位下全部技能", required = false, example = "101") + @Positive(message = "categoryId must be positive") + @RequestParam(required = false) Long categoryId, + HttpServletRequest request) { + Long userId = resolveAuthenticatedStudent(); + SkillLearningPathResponse data = skillLearningPathService.generate( + userId, industryId, occupationId, categoryId); + return ResponseEntity.ok(ApiResponse.ok(data, getRequestId(request))); + } + + private Long resolveAuthenticatedStudent() { + Authentication authentication = SecurityContextHolder.getContext().getAuthentication(); + if (authentication == null + || !authentication.isAuthenticated() + || "anonymousUser".equals(authentication.getName())) { + throw new BusinessException(ErrorCode.UNAUTHORIZED); + } + try { + Long userId = Long.valueOf(authentication.getName()); + profileService.assertStudentRole(userId); + return userId; + } catch (NumberFormatException exception) { + throw new BusinessException(ErrorCode.UNAUTHORIZED); + } + } + + private String getRequestId(HttpServletRequest request) { + return request.getHeader("X-Request-Id"); + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudQueryService.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudQueryService.java new file mode 100644 index 0000000..f2ae2e5 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillCloudQueryService.java @@ -0,0 +1,160 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.common.exception.BusinessException; +import com.mydreamjob.common.exception.ErrorCode; +import com.mydreamjob.skill.domain.SkillCategory; +import com.mydreamjob.skill.domain.SkillIndustry; +import com.mydreamjob.skill.domain.SkillOccupation; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillCloudNode; +import com.mydreamjob.skill.dto.SkillCloudResponse; +import com.mydreamjob.skill.dto.SkillCloudSkillProjection; +import com.mydreamjob.skill.repository.OccupationSkillRepository; +import com.mydreamjob.skill.repository.SkillCategoryRepository; +import com.mydreamjob.skill.repository.SkillIndustryRepository; +import com.mydreamjob.skill.repository.SkillOccupationRepository; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +import java.math.BigDecimal; +import java.util.Comparator; +import java.util.List; +import java.util.stream.IntStream; + +@Service +@Transactional(readOnly = true) +public class SkillCloudQueryService { + + private static final Comparator SKILL_ORDER = (left, right) -> { + int heatComparison = compareHeat(left.importance(), right.importance()); + if (heatComparison != 0) { + return heatComparison; + } + + int nameComparison = compareNullable(left.nameZh(), right.nameZh()); + if (nameComparison != 0) { + return nameComparison; + } + + return compareNullable(left.skillId(), right.skillId()); + }; + + private final SkillIndustryRepository industryRepository; + private final SkillOccupationRepository occupationRepository; + private final SkillCategoryRepository categoryRepository; + private final OccupationSkillRepository occupationSkillRepository; + + public SkillCloudQueryService(SkillIndustryRepository industryRepository, + SkillOccupationRepository occupationRepository, + SkillCategoryRepository categoryRepository, + OccupationSkillRepository occupationSkillRepository) { + this.industryRepository = industryRepository; + this.occupationRepository = occupationRepository; + this.categoryRepository = categoryRepository; + this.occupationSkillRepository = occupationSkillRepository; + } + + public SkillCloudResponse query(Long industryId, Long occupationId, Long categoryId) { + long validatedIndustryId = requirePositive(industryId, "industryId"); + long validatedOccupationId = requirePositive(occupationId, "occupationId"); + Long validatedCategoryId = categoryId == null ? null : requirePositive(categoryId, "categoryId"); + + SkillIndustry industry = industryRepository.findById(validatedIndustryId) + .orElseThrow(() -> badRequest("industry not found: industryId=" + validatedIndustryId)); + SkillOccupation occupation = occupationRepository.findById(validatedOccupationId) + .orElseThrow(() -> badRequest("occupation not found: occupationId=" + validatedOccupationId)); + if (!occupation.getIndustryId().equals(validatedIndustryId)) { + throw badRequest("occupation does not belong to industry: occupationId=" + + validatedOccupationId + ", industryId=" + validatedIndustryId); + } + + SkillCategory category = null; + if (validatedCategoryId != null) { + category = categoryRepository.findById(validatedCategoryId) + .orElseThrow(() -> badRequest("category not found: categoryId=" + validatedCategoryId)); + if (!category.getOccupationId().equals(validatedOccupationId)) { + throw badRequest("category does not belong to occupation: categoryId=" + + validatedCategoryId + ", occupationId=" + validatedOccupationId); + } + } + + List projections = validatedCategoryId == null + ? occupationSkillRepository.findSkillProjectionsByOccupationId(validatedOccupationId) + : occupationSkillRepository.findSkillProjectionsByOccupationIdAndCategoryId( + validatedOccupationId, validatedCategoryId); + + List sortedProjections = projections.stream() + .sorted(SKILL_ORDER) + .toList(); + List items = IntStream.range(0, sortedProjections.size()) + .mapToObj(index -> toItem(sortedProjections.get(index), index + 1)) + .toList(); + + return new SkillCloudResponse( + toNode(industry.getId(), industry.getCode(), industry.getNameZh()), + toNode(occupation.getId(), occupation.getCode(), occupation.getNameZh()), + category == null ? null : toNode(category.getId(), category.getCode(), category.getNameZh()), + items.size(), + items + ); + } + + private SkillCloudItem toItem(SkillCloudSkillProjection projection, int rank) { + return new SkillCloudItem( + rank, + projection.skillId(), + projection.skillCode(), + projection.nameZh(), + projection.nameEn(), + projection.sourceType(), + projection.sourceCode(), + projection.categoryId(), + projection.categoryName(), + projection.importance(), + projection.importanceSourceCode(), + projection.importanceElementId(), + projection.importanceIsProxy() + ); + } + + private static int compareHeat(BigDecimal left, BigDecimal right) { + if (left == null && right == null) { + return 0; + } + if (left == null) { + return 1; + } + if (right == null) { + return -1; + } + return right.compareTo(left); + } + + private static > int compareNullable(T left, T right) { + if (left == null && right == null) { + return 0; + } + if (left == null) { + return 1; + } + if (right == null) { + return -1; + } + return left.compareTo(right); + } + + private static SkillCloudNode toNode(Long id, String code, String name) { + return new SkillCloudNode(id, code, name); + } + + private static long requirePositive(Long value, String parameterName) { + if (value == null || value <= 0) { + throw badRequest(parameterName + " must be a positive integer"); + } + return value; + } + + private static BusinessException badRequest(String detail) { + return new BusinessException(ErrorCode.BAD_REQUEST, detail); + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillLearningPathService.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillLearningPathService.java new file mode 100644 index 0000000..bac87b6 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillLearningPathService.java @@ -0,0 +1,95 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.skill.domain.SkillCategory; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillLearningPathItem; +import com.mydreamjob.skill.dto.SkillLearningPathResponse; +import com.mydreamjob.skill.dto.SkillMatchResponse; +import com.mydreamjob.skill.repository.SkillCategoryRepository; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +import java.util.List; +import java.util.Locale; +import java.util.stream.IntStream; + +@Service +@Transactional(readOnly = true) +public class SkillLearningPathService { + + private final SkillMatchService skillMatchService; + private final SkillCategoryRepository skillCategoryRepository; + + public SkillLearningPathService(SkillMatchService skillMatchService, + SkillCategoryRepository skillCategoryRepository) { + this.skillMatchService = skillMatchService; + this.skillCategoryRepository = skillCategoryRepository; + } + + public SkillLearningPathResponse generate(Long studentUserId, + Long industryId, + Long occupationId, + Long categoryId) { + SkillMatchResponse matchResponse = skillMatchService.match( + studentUserId, industryId, occupationId, categoryId); + List missingSkills = matchResponse.missingSkills(); + List items = IntStream.range(0, missingSkills.size()) + .mapToObj(index -> toItem(missingSkills.get(index), index + 1, occupationId)) + .toList(); + + return new SkillLearningPathResponse(items.size(), items); + } + + private SkillLearningPathItem toItem(SkillCloudItem skill, int priority, Long occupationId) { + SuggestionTemplate template = resolveTemplate(skill, occupationId); + String skillName = skill.nameZh(); + return new SkillLearningPathItem( + priority, + skill.skillId(), + skillName, + skill.categoryId(), + skill.categoryName(), + template.render(skillName) + ); + } + + private SuggestionTemplate resolveTemplate(SkillCloudItem skill, Long occupationId) { + if (skill.categoryId() == null) { + return SuggestionTemplate.GENERAL; + } + + String templateCode = skillCategoryRepository + .findByIdAndOccupationId(skill.categoryId(), occupationId) + .map(SkillCategory::getSuggestionTemplateCode) + .orElse(null); + return SuggestionTemplate.fromCode(templateCode); + } + + private enum SuggestionTemplate { + PROGRAMMING_LANGUAGE("先掌握{skill}的基础语法和常见数据结构，再完成至少一个编码练习。"), + FRAMEWORK("先理解{skill}的核心概念与生命周期，再通过一个小型项目练习集成。"), + TOOL("先完成{skill}的安装与基础配置，再结合现有工程完成一次实际操作。"), + GENERAL("先了解{skill}的核心概念和常见应用场景，再通过一个简短实践巩固。"); + + private final String template; + + SuggestionTemplate(String template) { + this.template = template; + } + + String render(String skillName) { + return template.replace("{skill}", skillName); + } + + static SuggestionTemplate fromCode(String code) { + if (code == null || code.isBlank()) { + return GENERAL; + } + try { + return valueOf(code.trim().toUpperCase(Locale.ROOT)); + } catch (IllegalArgumentException exception) { + return GENERAL; + } + } + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillMatchService.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillMatchService.java new file mode 100644 index 0000000..f4b6103 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillMatchService.java @@ -0,0 +1,152 @@ +package com.mydreamjob.skill; + +import com.fasterxml.jackson.core.type.TypeReference; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.mydreamjob.profile.StudentProfile; +import com.mydreamjob.profile.StudentProfileRepository; +import com.mydreamjob.skill.domain.SkillAlias; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillCloudResponse; +import com.mydreamjob.skill.dto.SkillMatchResponse; +import com.mydreamjob.skill.repository.SkillAliasRepository; +import lombok.extern.slf4j.Slf4j; +import org.springframework.stereotype.Service; +import org.springframework.transaction.annotation.Transactional; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +@Slf4j +@Service +@Transactional(readOnly = true) +public class SkillMatchService { + + private static final int TOP_LIMIT = 10; + + private final SkillCloudQueryService skillCloudQueryService; + private final StudentProfileRepository studentProfileRepository; + private final SkillAliasRepository skillAliasRepository; + private final SkillNameNormalizer skillNameNormalizer; + private final ObjectMapper objectMapper; + + public SkillMatchService(SkillCloudQueryService skillCloudQueryService, + StudentProfileRepository studentProfileRepository, + SkillAliasRepository skillAliasRepository, + SkillNameNormalizer skillNameNormalizer, + ObjectMapper objectMapper) { + this.skillCloudQueryService = skillCloudQueryService; + this.studentProfileRepository = studentProfileRepository; + this.skillAliasRepository = skillAliasRepository; + this.skillNameNormalizer = skillNameNormalizer; + this.objectMapper = objectMapper; + } + + public SkillMatchResponse match(Long studentUserId, + Long industryId, + Long occupationId, + Long categoryId) { + SkillCloudResponse cloud = skillCloudQueryService.query(industryId, occupationId, categoryId); + int actualTopSize = Math.min(TOP_LIMIT, cloud.items().size()); + List topSkills = cloud.items().subList(0, actualTopSize); + Set studentSkills = loadStudentSkills(studentUserId); + Map> aliasesBySkillId = loadAliases(topSkills); + + List matchedSkills = new ArrayList<>(); + List missingSkills = new ArrayList<>(); + for (SkillCloudItem skill : topSkills) { + if (matches(skill, studentSkills, aliasesBySkillId.getOrDefault(skill.skillId(), List.of()))) { + matchedSkills.add(skill); + } else { + missingSkills.add(skill); + } + } + + int matchScore = actualTopSize == 0 + ? 0 + : (int) Math.round(matchedSkills.size() * 100.0 / actualTopSize); + return new SkillMatchResponse( + matchScore, + matchedSkills.size(), + actualTopSize, + matchedSkills, + missingSkills + ); + } + + private Set loadStudentSkills(Long studentUserId) { + if (studentUserId == null) { + return Set.of(); + } + + StudentProfile profile = studentProfileRepository.findByUserId(studentUserId).orElse(null); + if (profile == null || profile.getSkillsJson() == null || profile.getSkillsJson().isBlank()) { + return Set.of(); + } + + try { + List parsed = objectMapper.readValue( + profile.getSkillsJson(), + new TypeReference>() {} + ); + if (parsed == null || parsed.isEmpty()) { + return Set.of(); + } + + Set normalizedSkills = new HashSet<>(); + for (String skill : parsed) { + if (skill == null) { + continue; + } + String normalized = skillNameNormalizer.normalize(skill); + if (!normalized.isBlank()) { + normalizedSkills.add(normalized); + } + } + return Set.copyOf(normalizedSkills); + } catch (Exception exception) { + log.warn("Failed to parse student skills JSON: profileId={}, errorType={}", + profile.getId(), exception.getClass().getSimpleName()); + return Set.of(); + } + } + + private Map> loadAliases(List topSkills) { + Collection skillIds = topSkills.stream() + .map(SkillCloudItem::skillId) + .filter(java.util.Objects::nonNull) + .toList(); + if (skillIds.isEmpty()) { + return Map.of(); + } + + Map> aliasesBySkillId = new HashMap<>(); + for (SkillAlias alias : skillAliasRepository.findAllBySkillIdInOrderByIdAsc(skillIds)) { + aliasesBySkillId.computeIfAbsent(alias.getSkillId(), ignored -> new ArrayList<>()).add(alias); + } + return aliasesBySkillId; + } + + private boolean matches(SkillCloudItem skill, + Set studentSkills, + List aliases) { + if (containsNormalized(studentSkills, skill.nameZh()) + || containsNormalized(studentSkills, skill.nameEn()) + || containsNormalized(studentSkills, skill.skillCode())) { + return true; + } + return aliases.stream().anyMatch(alias -> containsNormalized(studentSkills, alias.getAlias())); + } + + private boolean containsNormalized(Set studentSkills, String candidate) { + if (candidate == null) { + return false; + } + String normalized = skillNameNormalizer.normalize(candidate); + return !normalized.isBlank() && studentSkills.contains(normalized); + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillNameNormalizer.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillNameNormalizer.java new file mode 100644 index 0000000..84ddf2e --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/SkillNameNormalizer.java @@ -0,0 +1,28 @@ +package com.mydreamjob.skill; + +import org.springframework.stereotype.Component; + +import java.text.Normalizer; +import java.util.Locale; +import java.util.regex.Pattern; + +@Component +public class SkillNameNormalizer { + + private static final Pattern UNICODE_HYPHENS = Pattern.compile("\\p{Pd}"); + private static final Pattern WHITESPACE = Pattern.compile("\\s+", Pattern.UNICODE_CHARACTER_CLASS); + + public String normalize(String value) { + if (value == null) { + return ""; + } + + String normalized = Normalizer.normalize(value, Normalizer.Form.NFKC) + .trim() + .toLowerCase(Locale.ROOT); + normalized = UNICODE_HYPHENS.matcher(normalized).replaceAll(" "); + normalized = normalized.replace(".", ""); + normalized = WHITESPACE.matcher(normalized).replaceAll(" "); + return normalized.trim(); + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/OccupationSkill.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/OccupationSkill.java new file mode 100644 index 0000000..38046c6 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/OccupationSkill.java @@ -0,0 +1,62 @@ +package com.mydreamjob.skill.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.math.BigDecimal; +import java.time.LocalDateTime; + +@Getter +@Setter +@Builder +@NoArgsConstructor +@AllArgsConstructor +@Entity +@Table(name = "occupation_skills") +public class OccupationSkill { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(name = "occupation_id", nullable = false) + private Long occupationId; + + @Column(name = "skill_id", nullable = false) + private Long skillId; + + @Column(name = "category_id", nullable = false) + private Long categoryId; + + @Column(name = "importance", precision = 4, scale = 2) + private BigDecimal importance; + + @Column(name = "importance_source_code", length = 20) + private String importanceSourceCode; + + @Column(name = "importance_element_id", length = 30) + private String importanceElementId; + @Column(name = "importance_is_proxy", nullable = false) + private Boolean importanceIsProxy; + + @Column(name = "relationship_type", nullable = false, length = 20) + private String relationshipType; + + @Column(name = "source_version_id", nullable = false) + private Long sourceVersionId; + + @Column(name = "created_at", nullable = false, updatable = false) + private LocalDateTime createdAt; + + @Column(name = "updated_at", nullable = false) + private LocalDateTime updatedAt; +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/Skill.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/Skill.java new file mode 100644 index 0000000..0d84071 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/Skill.java @@ -0,0 +1,56 @@ +package com.mydreamjob.skill.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.time.LocalDateTime; + +@Getter +@Setter +@Builder +@NoArgsConstructor +@AllArgsConstructor +@Entity +@Table(name = "skills") +public class Skill { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(name = "code", nullable = false, length = 120) + private String code; + + @Column(name = "name_zh", nullable = false, length = 180) + private String nameZh; + + @Column(name = "name_en", length = 180) + private String nameEn; + + @Column(name = "source_type", nullable = false, length = 20) + private String sourceType; + + @Column(name = "source_code", nullable = false, length = 180) + private String sourceCode; + + @Column(name = "source_version_id", nullable = false) + private Long sourceVersionId; + + @Column(name = "sort_order", nullable = false) + private Integer sortOrder; + + @Column(name = "created_at", nullable = false, updatable = false) + private LocalDateTime createdAt; + + @Column(name = "updated_at", nullable = false) + private LocalDateTime updatedAt; +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillAlias.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillAlias.java new file mode 100644 index 0000000..0d05f04 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillAlias.java @@ -0,0 +1,41 @@ +package com.mydreamjob.skill.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.time.LocalDateTime; + +@Getter +@Setter +@Builder +@NoArgsConstructor +@AllArgsConstructor +@Entity +@Table(name = "skill_aliases") +public class SkillAlias { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(name = "skill_id", nullable = false) + private Long skillId; + + @Column(name = "alias", nullable = false, length = 180) + private String alias; + + @Column(name = "normalized_alias", nullable = false, length = 180) + private String normalizedAlias; + + @Column(name = "created_at", nullable = false, updatable = false) + private LocalDateTime createdAt; +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillCategory.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillCategory.java new file mode 100644 index 0000000..dd58b08 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillCategory.java @@ -0,0 +1,50 @@ +package com.mydreamjob.skill.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.time.LocalDateTime; + +@Getter +@Setter +@Builder +@NoArgsConstructor +@AllArgsConstructor +@Entity +@Table(name = "skill_categories") +public class SkillCategory { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(name = "occupation_id", nullable = false) + private Long occupationId; + + @Column(name = "code", nullable = false, length = 100) + private String code; + + @Column(name = "name_zh", nullable = false, length = 120) + private String nameZh; + + @Column(name = "suggestion_template_code", nullable = false, length = 30) + private String suggestionTemplateCode; + + @Column(name = "sort_order", nullable = false) + private Integer sortOrder; + + @Column(name = "created_at", nullable = false, updatable = false) + private LocalDateTime createdAt; + + @Column(name = "updated_at", nullable = false) + private LocalDateTime updatedAt; +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillIndustry.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillIndustry.java new file mode 100644 index 0000000..7fbd239 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillIndustry.java @@ -0,0 +1,47 @@ +package com.mydreamjob.skill.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.time.LocalDateTime; + +@Getter +@Setter +@Builder +@NoArgsConstructor +@AllArgsConstructor +@Entity +@Table(name = "skill_industries") +public class SkillIndustry { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(name = "code", nullable = false, length = 80) + private String code; + + @Column(name = "name_zh", nullable = false, length = 120) + private String nameZh; + + @Column(name = "source_code", length = 120) + private String sourceCode; + + @Column(name = "sort_order", nullable = false) + private Integer sortOrder; + + @Column(name = "created_at", nullable = false, updatable = false) + private LocalDateTime createdAt; + + @Column(name = "updated_at", nullable = false) + private LocalDateTime updatedAt; +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillOccupation.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillOccupation.java new file mode 100644 index 0000000..4a83dac --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillOccupation.java @@ -0,0 +1,56 @@ +package com.mydreamjob.skill.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.time.LocalDateTime; + +@Getter +@Setter +@Builder +@NoArgsConstructor +@AllArgsConstructor +@Entity +@Table(name = "skill_occupations") +public class SkillOccupation { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(name = "industry_id", nullable = false) + private Long industryId; + + @Column(name = "code", nullable = false, length = 100) + private String code; + + @Column(name = "name_zh", nullable = false, length = 150) + private String nameZh; + + @Column(name = "name_en", length = 180) + private String nameEn; + + @Column(name = "source_code", nullable = false, length = 150) + private String sourceCode; + + @Column(name = "source_version_id", nullable = false) + private Long sourceVersionId; + + @Column(name = "sort_order", nullable = false) + private Integer sortOrder; + + @Column(name = "created_at", nullable = false, updatable = false) + private LocalDateTime createdAt; + + @Column(name = "updated_at", nullable = false) + private LocalDateTime updatedAt; +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillSourceVersion.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillSourceVersion.java new file mode 100644 index 0000000..498cebd --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/domain/SkillSourceVersion.java @@ -0,0 +1,50 @@ +package com.mydreamjob.skill.domain; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Getter; +import lombok.NoArgsConstructor; +import lombok.Setter; + +import java.time.LocalDateTime; + +@Getter +@Setter +@Builder +@NoArgsConstructor +@AllArgsConstructor +@Entity +@Table(name = "skill_source_versions") +public class SkillSourceVersion { + + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @Column(name = "source_name", nullable = false, length = 100) + private String sourceName; + + @Column(name = "source_version", nullable = false, length = 150) + private String sourceVersion; + + @Column(name = "license", nullable = false, length = 255) + private String license; + + @Column(name = "attribution_url", nullable = false, length = 500) + private String attributionUrl; + + @Column(name = "imported_at", nullable = false) + private LocalDateTime importedAt; + + @Column(name = "created_at", nullable = false, updatable = false) + private LocalDateTime createdAt; + + @Column(name = "updated_at", nullable = false) + private LocalDateTime updatedAt; +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudItem.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudItem.java new file mode 100644 index 0000000..b41a71d --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudItem.java @@ -0,0 +1,20 @@ +package com.mydreamjob.skill.dto; + +import java.math.BigDecimal; + +public record SkillCloudItem( + int rank, + Long skillId, + String skillCode, + String nameZh, + String nameEn, + String sourceType, + String sourceCode, + Long categoryId, + String categoryName, + BigDecimal heat, + String heatSourceCode, + String heatElementId, + Boolean heatProxy +) { +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudNode.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudNode.java new file mode 100644 index 0000000..55a28fd --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudNode.java @@ -0,0 +1,8 @@ +package com.mydreamjob.skill.dto; + +public record SkillCloudNode( + Long id, + String code, + String name +) { +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudResponse.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudResponse.java new file mode 100644 index 0000000..326076e --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudResponse.java @@ -0,0 +1,15 @@ +package com.mydreamjob.skill.dto; + +import java.util.List; + +public record SkillCloudResponse( + SkillCloudNode industry, + SkillCloudNode occupation, + SkillCloudNode category, + int total, + List items +) { + public SkillCloudResponse { + items = List.copyOf(items); + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudSkillProjection.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudSkillProjection.java new file mode 100644 index 0000000..d2d3619 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillCloudSkillProjection.java @@ -0,0 +1,21 @@ +package com.mydreamjob.skill.dto; + +import java.math.BigDecimal; + +public record SkillCloudSkillProjection( + Long skillId, + String skillCode, + String nameZh, + String nameEn, + String sourceType, + String sourceCode, + Long categoryId, + String categoryName, + BigDecimal importance, + String importanceSourceCode, + String importanceElementId, + Boolean importanceIsProxy, + Integer skillSortOrder, + Integer categorySortOrder +) { +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillLearningPathItem.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillLearningPathItem.java new file mode 100644 index 0000000..28b6935 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillLearningPathItem.java @@ -0,0 +1,11 @@ +package com.mydreamjob.skill.dto; + +public record SkillLearningPathItem( + int priority, + Long skillId, + String skillName, + Long categoryId, + String categoryName, + String suggestion +) { +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillLearningPathResponse.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillLearningPathResponse.java new file mode 100644 index 0000000..11ce18c --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillLearningPathResponse.java @@ -0,0 +1,12 @@ +package com.mydreamjob.skill.dto; + +import java.util.List; + +public record SkillLearningPathResponse( + int total, + List items +) { + public SkillLearningPathResponse { + items = List.copyOf(items); + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillMatchResponse.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillMatchResponse.java new file mode 100644 index 0000000..25953ea --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/dto/SkillMatchResponse.java @@ -0,0 +1,16 @@ +package com.mydreamjob.skill.dto; + +import java.util.List; + +public record SkillMatchResponse( + int matchScore, + int matchedCount, + int totalCount, + List matchedSkills, + List missingSkills +) { + public SkillMatchResponse { + matchedSkills = List.copyOf(matchedSkills); + missingSkills = List.copyOf(missingSkills); + } +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/OccupationSkillRepository.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/OccupationSkillRepository.java new file mode 100644 index 0000000..d3c0de1 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/OccupationSkillRepository.java @@ -0,0 +1,73 @@ +package com.mydreamjob.skill.repository; + +import com.mydreamjob.skill.domain.OccupationSkill; +import com.mydreamjob.skill.dto.SkillCloudSkillProjection; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; +import org.springframework.transaction.annotation.Transactional; + +import java.util.List; + +public interface OccupationSkillRepository extends JpaRepository { + + long countByOccupationId(Long occupationId); + + long countByOccupationIdAndCategoryId(Long occupationId, Long categoryId); + + @Transactional(readOnly = true) + @Query(""" + SELECT new com.mydreamjob.skill.dto.SkillCloudSkillProjection( + os.skillId, + s.code, + s.nameZh, + s.nameEn, + s.sourceType, + s.sourceCode, + c.id, + c.nameZh, + os.importance, + os.importanceSourceCode, + os.importanceElementId, + os.importanceIsProxy, + s.sortOrder, + c.sortOrder + ) + FROM OccupationSkill os, Skill s, SkillCategory c + WHERE os.skillId = s.id + AND os.categoryId = c.id + AND c.occupationId = os.occupationId + AND os.occupationId = :occupationId + """) + List findSkillProjectionsByOccupationId( + @Param("occupationId") Long occupationId); + + @Transactional(readOnly = true) + @Query(""" + SELECT new com.mydreamjob.skill.dto.SkillCloudSkillProjection( + os.skillId, + s.code, + s.nameZh, + s.nameEn, + s.sourceType, + s.sourceCode, + c.id, + c.nameZh, + os.importance, + os.importanceSourceCode, + os.importanceElementId, + os.importanceIsProxy, + s.sortOrder, + c.sortOrder + ) + FROM OccupationSkill os, Skill s, SkillCategory c + WHERE os.skillId = s.id + AND os.categoryId = c.id + AND c.occupationId = os.occupationId + AND os.occupationId = :occupationId + AND os.categoryId = :categoryId + """) + List findSkillProjectionsByOccupationIdAndCategoryId( + @Param("occupationId") Long occupationId, + @Param("categoryId") Long categoryId); +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillAliasRepository.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillAliasRepository.java new file mode 100644 index 0000000..f674913 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillAliasRepository.java @@ -0,0 +1,15 @@ +package com.mydreamjob.skill.repository; + +import com.mydreamjob.skill.domain.SkillAlias; +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.Collection; +import java.util.List; +import java.util.Optional; + +public interface SkillAliasRepository extends JpaRepository { + + Optional findByNormalizedAlias(String normalizedAlias); + + List findAllBySkillIdInOrderByIdAsc(Collection skillIds); +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillCategoryRepository.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillCategoryRepository.java new file mode 100644 index 0000000..e83f478 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillCategoryRepository.java @@ -0,0 +1,13 @@ +package com.mydreamjob.skill.repository; + +import com.mydreamjob.skill.domain.SkillCategory; +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.Optional; + +public interface SkillCategoryRepository extends JpaRepository { + + boolean existsByIdAndOccupationId(Long id, Long occupationId); + + Optional findByIdAndOccupationId(Long id, Long occupationId); +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillIndustryRepository.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillIndustryRepository.java new file mode 100644 index 0000000..fc80d54 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillIndustryRepository.java @@ -0,0 +1,7 @@ +package com.mydreamjob.skill.repository; + +import com.mydreamjob.skill.domain.SkillIndustry; +import org.springframework.data.jpa.repository.JpaRepository; + +public interface SkillIndustryRepository extends JpaRepository { +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillOccupationRepository.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillOccupationRepository.java new file mode 100644 index 0000000..172b22c --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillOccupationRepository.java @@ -0,0 +1,13 @@ +package com.mydreamjob.skill.repository; + +import com.mydreamjob.skill.domain.SkillOccupation; +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.Optional; + +public interface SkillOccupationRepository extends JpaRepository { + + boolean existsByIdAndIndustryId(Long id, Long industryId); + + Optional findByIdAndIndustryId(Long id, Long industryId); +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillRepository.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillRepository.java new file mode 100644 index 0000000..507f7a6 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillRepository.java @@ -0,0 +1,15 @@ +package com.mydreamjob.skill.repository; + +import com.mydreamjob.skill.domain.Skill; +import org.springframework.data.jpa.repository.JpaRepository; + +import java.util.Collection; +import java.util.List; +import java.util.Optional; + +public interface SkillRepository extends JpaRepository { + + Optional findByCode(String code); + + List findAllByIdInOrderByIdAsc(Collection ids); +} diff --git a/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillSourceVersionRepository.java b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillSourceVersionRepository.java new file mode 100644 index 0000000..a22a7a4 --- /dev/null +++ b/mydreamjob-server/src/main/java/com/mydreamjob/skill/repository/SkillSourceVersionRepository.java @@ -0,0 +1,7 @@ +package com.mydreamjob.skill.repository; + +import com.mydreamjob.skill.domain.SkillSourceVersion; +import org.springframework.data.jpa.repository.JpaRepository; + +public interface SkillSourceVersionRepository extends JpaRepository { +} diff --git a/mydreamjob-server/src/main/resources/db/migration/V35__skill_cloud_taxonomy.sql b/mydreamjob-server/src/main/resources/db/migration/V35__skill_cloud_taxonomy.sql new file mode 100644 index 0000000..db81f93 --- /dev/null +++ b/mydreamjob-server/src/main/resources/db/migration/V35__skill_cloud_taxonomy.sql @@ -0,0 +1,120 @@ +-- Flyway V35: 行业技能词云知识库表结构 +-- 来源与许可：ESCO v1.2.1 提供职位/技能/分类关系，O*NET 31.0 提供 Importance。 +-- 本迁移只建立结构，代表性种子数据由 V36 导入。 + +CREATE TABLE skill_source_versions ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + source_name VARCHAR(100) NOT NULL COMMENT '来源名称：ESCO/O*NET', + source_version VARCHAR(150) NOT NULL COMMENT '固定来源版本', + license VARCHAR(255) NOT NULL COMMENT '来源许可证或复用条件', + attribution_url VARCHAR(500) NOT NULL COMMENT '官方来源或归属地址', + imported_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '导入时间', + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + CONSTRAINT uk_skill_source_versions_source UNIQUE (source_name, source_version) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='技能词库来源版本'; + +CREATE TABLE skill_industries ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + code VARCHAR(80) NOT NULL COMMENT '项目行业稳定编码', + name_zh VARCHAR(120) NOT NULL COMMENT '中文行业名称', + source_code VARCHAR(120) COMMENT '上游或项目映射来源编码', + sort_order INT NOT NULL DEFAULT 0 COMMENT '展示顺序', + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + CONSTRAINT uk_skill_industries_code UNIQUE (code) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='技能词云行业'; + +CREATE TABLE skill_occupations ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + industry_id BIGINT NOT NULL COMMENT '所属行业', + code VARCHAR(100) NOT NULL COMMENT '职位稳定编码', + name_zh VARCHAR(150) NOT NULL COMMENT '中文职位名称', + name_en VARCHAR(180) COMMENT '英文职位名称', + source_code VARCHAR(150) NOT NULL COMMENT 'ESCO 职位来源 ID', + source_version_id BIGINT NOT NULL COMMENT '来源版本', + sort_order INT NOT NULL DEFAULT 0 COMMENT '展示顺序', + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + CONSTRAINT uk_skill_occupations_industry_code UNIQUE (industry_id, code), + CONSTRAINT fk_skill_occupations_industry FOREIGN KEY (industry_id) REFERENCES skill_industries (id), + CONSTRAINT fk_skill_occupations_source_version FOREIGN KEY (source_version_id) REFERENCES skill_source_versions (id) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='技能词云职位'; + +CREATE TABLE skill_categories ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + occupation_id BIGINT NOT NULL COMMENT '所属职位', + code VARCHAR(100) NOT NULL COMMENT '技能分类稳定编码', + name_zh VARCHAR(120) NOT NULL COMMENT '中文分类名称', + suggestion_template_code VARCHAR(30) NOT NULL COMMENT '学习建议模板编码', + sort_order INT NOT NULL DEFAULT 0 COMMENT '展示顺序', + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + CONSTRAINT uk_skill_categories_occupation_code UNIQUE (occupation_id, code), + CONSTRAINT ck_skill_categories_suggestion_template CHECK ( + suggestion_template_code IN ('PROGRAMMING_LANGUAGE', 'FRAMEWORK', 'TOOL', 'GENERAL') + ), + CONSTRAINT fk_skill_categories_occupation FOREIGN KEY (occupation_id) REFERENCES skill_occupations (id) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='职位技能分类'; + +CREATE TABLE skills ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + code VARCHAR(120) NOT NULL COMMENT '项目技能稳定编码', + name_zh VARCHAR(180) NOT NULL COMMENT '中文标准技能名称', + name_en VARCHAR(180) COMMENT '英文标准技能名称', + source_type VARCHAR(20) NOT NULL COMMENT '来源类型', + source_code VARCHAR(180) NOT NULL COMMENT 'ESCO/O*NET 或项目来源 ID', + source_version_id BIGINT NOT NULL COMMENT '来源版本', + sort_order INT NOT NULL DEFAULT 0 COMMENT '展示顺序', + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + CONSTRAINT uk_skills_code UNIQUE (code), + CONSTRAINT uk_skills_source UNIQUE (source_type, source_code), + CONSTRAINT ck_skills_source_type CHECK (source_type IN ('ESCO', 'O*NET', 'PROJECT')), + CONSTRAINT fk_skills_source_version FOREIGN KEY (source_version_id) REFERENCES skill_source_versions (id) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='标准技能'; + +CREATE TABLE skill_aliases ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + skill_id BIGINT NOT NULL COMMENT '标准技能', + alias VARCHAR(180) NOT NULL COMMENT '技能别名', + normalized_alias VARCHAR(180) NOT NULL COMMENT '归一化技能别名', + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + CONSTRAINT uk_skill_aliases_normalized_alias UNIQUE (normalized_alias), + CONSTRAINT uk_skill_aliases_skill_alias UNIQUE (skill_id, alias), + CONSTRAINT fk_skill_aliases_skill FOREIGN KEY (skill_id) REFERENCES skills (id) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='标准技能别名'; + +CREATE TABLE occupation_skills ( + id BIGINT AUTO_INCREMENT PRIMARY KEY, + occupation_id BIGINT NOT NULL COMMENT '职位', + skill_id BIGINT NOT NULL COMMENT '标准技能', + category_id BIGINT NOT NULL COMMENT '职位内技能分类', + importance DECIMAL(4,2) COMMENT 'O*NET Importance，0-5，允许为空', + importance_source_code VARCHAR(20) COMMENT 'O*NET SOC 来源，非空热度必填', + importance_element_id VARCHAR(30) COMMENT 'O*NET Element ID，非空热度必填', + importance_is_proxy TINYINT(1) NOT NULL DEFAULT 0 COMMENT '是否为项目分类到 O*NET 元素的代理映射', + relationship_type VARCHAR(20) NOT NULL COMMENT '职位技能关系类型', + source_version_id BIGINT NOT NULL COMMENT 'Importance 来源版本', + created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, + updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + CONSTRAINT uk_occupation_skills_occupation_skill UNIQUE (occupation_id, skill_id), + CONSTRAINT ck_occupation_skills_importance CHECK ( + importance IS NULL OR (importance >= 0.00 AND importance <= 5.00) + ), + CONSTRAINT ck_occupation_skills_importance_provenance CHECK ( + importance IS NULL OR (importance_source_code IS NOT NULL AND importance_element_id IS NOT NULL) + ), + CONSTRAINT ck_occupation_skills_importance_proxy CHECK ( + importance_is_proxy IN (0, 1) + ), + CONSTRAINT ck_occupation_skills_relationship_type CHECK ( + relationship_type IN ('ESSENTIAL', 'OPTIONAL', 'PROJECT') + ), + CONSTRAINT fk_occupation_skills_occupation FOREIGN KEY (occupation_id) REFERENCES skill_occupations (id), + CONSTRAINT fk_occupation_skills_skill FOREIGN KEY (skill_id) REFERENCES skills (id), + CONSTRAINT fk_occupation_skills_category FOREIGN KEY (category_id) REFERENCES skill_categories (id), + CONSTRAINT fk_occupation_skills_source_version FOREIGN KEY (source_version_id) REFERENCES skill_source_versions (id), + KEY idx_occupation_skills_occupation_category (occupation_id, category_id), + KEY idx_occupation_skills_occupation_importance (occupation_id, importance) +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='职位技能关系与热度'; diff --git a/mydreamjob-server/src/main/resources/db/migration/V36__skill_cloud_seed.sql b/mydreamjob-server/src/main/resources/db/migration/V36__skill_cloud_seed.sql new file mode 100644 index 0000000..e4a03a4 --- /dev/null +++ b/mydreamjob-server/src/main/resources/db/migration/V36__skill_cloud_seed.sql @@ -0,0 +1,413 @@ +-- Flyway V36: ESCO/O*NET 代表性行业技能种子数据 +-- 本迁移只导入互联网、金融、教育三类行业及 10 个代表性职位。 +-- ESCO 负责职位、通用技能和分类主关系；O*NET software_skills 仅补充 ESCO 缺失的具体技术。 +-- O*NET software_skills.source_code 格式：SOC|Workplace Example|Element ID，均来自本地 O*NET 31.0 CSV 的真实行。 +-- +-- ESCO 职位映射： +-- FRONTEND_DEVELOPER -> c40a2919-48a9-40ea-b506-1f34f693496d +-- BACKEND_DEVELOPER -> f2b15a0e-e65a-438a-affb-29b9d50b77d1 +-- QA_ENGINEER -> 106f79e4-6264-45f1-9e7a-297435cd684b +-- DEVOPS_ENGINEER -> 使用后端软件开发者 ESCO URI 作为项目代理（ESCO 无更近 DevOps 节点） +-- FINANCIAL_DATA_ANALYST -> 8d586ee9-0ab1-4155-a3b0-2ca786c8e48c +-- RISK_MODELING_ENGINEER -> b09acdda-6680-49d1-a411-129610ee3bc0 +-- QUANTITATIVE_DEVELOPER -> 使用软件开发者 ESCO URI 作为项目代理（ESCO 无更近量化开发节点） +-- EDUCATION_PRODUCT_MANAGER -> a79170ad-0b7b-4697-864f-1748359aa10a +-- INSTRUCTIONAL_DESIGNER -> a9c30651-ccbc-4a80-900c-7880615cdf6e +-- EDUCATION_DATA_ANALYST -> d3edb8f8-3a06-47a0-8fb9-9b212c006aa2 +-- +-- O*NET Importance 代理 SOC： +-- FRONTEND_DEVELOPER -> 15-1254.00 Web Developers +-- BACKEND_DEVELOPER -> 15-1252.00 Software Developers +-- QA_ENGINEER -> 15-1253.00 Software Quality Assurance Analysts and Testers +-- DEVOPS_ENGINEER -> 15-1244.00 Network and Computer Systems Administrators +-- FINANCIAL_DATA_ANALYST -> 13-2031.00 Budget Analysts（目标金融分析师 SOC 在本地 O*NET 31.0 Importance CSV 中无数据，故使用语义较近的预算分析师代理） +-- RISK_MODELING_ENGINEER -> 13-2053.00 Insurance Underwriters（目标金融风险专家 SOC 在本地 Importance CSV 中无数据，故使用包含风险决策活动的保险承保人代理） +-- QUANTITATIVE_DEVELOPER -> 13-2099.01 Financial Quantitative Analysts +-- EDUCATION_PRODUCT_MANAGER -> 13-1082.00 Project Management Specialists（无直接教育产品经理 SOC，使用项目管理专家代理） +-- INSTRUCTIONAL_DESIGNER -> 25-9031.00 Instructional Coordinators +-- EDUCATION_DATA_ANALYST -> 15-2051.00 Data Scientists +-- +-- 分类代理规则（importance_is_proxy=1）： +-- TECH=4.A.3.b.1 Working with Computers；PROCESS=4.A.2.a.2 Processing Information； +-- ANALYSIS=4.A.2.a.4 Analyzing Data or Information；COMPLIANCE=4.A.2.a.3 Evaluating Information to Determine Compliance with Standards； +-- DESIGN=4.A.2.b.2 Thinking Creatively；STRATEGY=4.A.2.b.4 Developing Objectives and Strategies； +-- MONITOR=4.A.1.a.2 Monitoring Processes, Materials, or Surroundings；ESTIMATE=4.A.1.b.3 Estimating the Quantifiable Characteristics of Products, Events, or Information； +-- COMM=4.A.4.a.2 Communicating with Supervisors, Peers, or Subordinates；EDU=2.C.6 Education and Training； +-- FINANCE=2.C.1.c Economics and Accounting；MATH=2.C.4.a Mathematics；SECURITY=2.C.8.a Public Safety and Security。 +-- 每条非空 importance 均保存上述代理 SOC、Element ID 和 importance_is_proxy=1，可由本地 O*NET CSV 逐条复核。 +-- 无法稳定映射 Importance 的技能保留关系并写 NULL，不伪造热度。 +-- +INSERT INTO skill_source_versions (id, source_name, source_version, license, attribution_url, imported_at) VALUES +(1, 'ESCO', '1.2.1 (Release v5.7.0, Last update 10/12/2025)', 'Commission Decision 2011/833/EU', 'https://esco.ec.europa.eu/en/use-esco/download', CURRENT_TIMESTAMP), +(2, 'O*NET', '31.0 Database', 'CC BY 4.0', 'https://www.onetcenter.org/database.html', CURRENT_TIMESTAMP); +INSERT INTO skill_industries (id, code, name_zh, source_code, sort_order) VALUES +(1, 'INTERNET', '互联网', 'PROJECT:INDUSTRY:INTERNET', 1), +(2, 'FINANCE', '金融', 'PROJECT:INDUSTRY:FINANCE', 2), +(3, 'EDUCATION', '教育', 'PROJECT:INDUSTRY:EDUCATION', 3); +-- DevOps 与量化开发没有更近的 ESCO 职位节点，source_code 显式复用软件开发者 URI；这是项目代理映射，不是上游原始职位关系。 +INSERT INTO skill_occupations (id, industry_id, code, name_zh, name_en, source_code, source_version_id, sort_order) VALUES +(1, 1, 'FRONTEND_DEVELOPER', '前端开发工程师', 'Frontend Developer', 'http://data.europa.eu/esco/occupation/c40a2919-48a9-40ea-b506-1f34f693496d', 1, 1), +(2, 1, 'BACKEND_DEVELOPER', '后端开发工程师', 'Backend Developer', 'http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1', 1, 2), +(3, 1, 'QA_ENGINEER', '测试工程师', 'Software QA Engineer', 'http://data.europa.eu/esco/occupation/106f79e4-6264-45f1-9e7a-297435cd684b', 1, 3), +(4, 1, 'DEVOPS_ENGINEER', 'DevOps 工程师', 'DevOps Engineer', 'http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1', 1, 4), +(5, 2, 'FINANCIAL_DATA_ANALYST', '金融数据分析师', 'Financial Data Analyst', 'http://data.europa.eu/esco/occupation/8d586ee9-0ab1-4155-a3b0-2ca786c8e48c', 1, 1), +(6, 2, 'RISK_MODELING_ENGINEER', '风控建模工程师', 'Risk Modeling Engineer', 'http://data.europa.eu/esco/occupation/b09acdda-6680-49d1-a411-129610ee3bc0', 1, 2), +(7, 2, 'QUANTITATIVE_DEVELOPER', '量化开发工程师', 'Quantitative Developer', 'http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1', 1, 3), +(8, 3, 'EDUCATION_PRODUCT_MANAGER', '教育产品经理', 'Education Product Manager', 'http://data.europa.eu/esco/occupation/a79170ad-0b7b-4697-864f-1748359aa10a', 1, 1), +(9, 3, 'INSTRUCTIONAL_DESIGNER', '教学设计师', 'Instructional Designer', 'http://data.europa.eu/esco/occupation/a9c30651-ccbc-4a80-900c-7880615cdf6e', 1, 2), +(10, 3, 'EDUCATION_DATA_ANALYST', '教育数据分析师', 'Education Data Analyst', 'http://data.europa.eu/esco/occupation/d3edb8f8-3a06-47a0-8fb9-9b212c006aa2', 1, 3); +INSERT INTO skill_categories (id, occupation_id, code, name_zh, suggestion_template_code, sort_order) VALUES +(101, 1, 'LANGUAGE', '编程语言', 'PROGRAMMING_LANGUAGE', 1), +(102, 1, 'WEB_BASIC', 'Web 基础', 'FRAMEWORK', 2), +(103, 1, 'FRAMEWORK', '前端框架', 'FRAMEWORK', 3), +(104, 1, 'TOOLS', '工程化工具', 'TOOL', 4), +(105, 1, 'BROWSER_DEBUG', '浏览器与调试', 'TOOL', 5), +(106, 1, 'TEST', '测试', 'GENERAL', 6), +(107, 1, 'COLLABORATION', '版本控制与协作', 'GENERAL', 7), +(201, 2, 'LANGUAGE', '编程语言', 'PROGRAMMING_LANGUAGE', 1), +(202, 2, 'FRAMEWORK', '后端框架与平台', 'FRAMEWORK', 2), +(203, 2, 'DATABASE', '数据库', 'TOOL', 3), +(204, 2, 'CACHE_MESSAGING', '缓存与消息', 'TOOL', 4), +(205, 2, 'CLOUD_DEVOPS', '云/DevOps', 'TOOL', 5), +(206, 2, 'TEST', '测试', 'GENERAL', 6), +(207, 2, 'TOOLS', '工程化工具', 'TOOL', 7), +(208, 2, 'SECURITY', '安全', 'GENERAL', 8), +(301, 3, 'LANGUAGE', '编程语言', 'PROGRAMMING_LANGUAGE', 1), +(302, 3, 'TEST_DESIGN', '测试设计', 'GENERAL', 2), +(303, 3, 'AUTOMATION_TEST', '自动化测试框架', 'FRAMEWORK', 3), +(304, 3, 'API_DATABASE_TEST', '接口与数据库测试', 'TOOL', 4), +(305, 3, 'PERFORMANCE_TEST', '性能测试', 'TOOL', 5), +(306, 3, 'CICD_TOOLS', 'CI/CD 与工程化工具', 'TOOL', 6), +(401, 4, 'LANGUAGE_SCRIPT', '编程语言与脚本', 'PROGRAMMING_LANGUAGE', 1), +(402, 4, 'CLOUD', '云平台', 'TOOL', 2), +(403, 4, 'CONTAINER', '容器与编排', 'TOOL', 3), +(404, 4, 'CICD', 'CI/CD', 'TOOL', 4), +(405, 4, 'IAC', '配置管理与 IaC', 'TOOL', 5), +(406, 4, 'OBSERVABILITY', '可观测性', 'TOOL', 6), +(407, 4, 'SECURITY', '安全', 'GENERAL', 7), +(408, 4, 'DATABASE', '数据库', 'TOOL', 8), +(501, 5, 'LANGUAGE', '编程语言', 'PROGRAMMING_LANGUAGE', 1), +(502, 5, 'ANALYSIS_VISUALIZATION', '数据分析与可视化', 'TOOL', 2), +(503, 5, 'DATABASE_WAREHOUSE', '数据库与数据仓库', 'TOOL', 3), +(504, 5, 'STATISTICS_MODELING', '统计与建模', 'FRAMEWORK', 4), +(505, 5, 'FINANCE_PLATFORM', '金融计算框架/平台', 'FRAMEWORK', 5), +(506, 5, 'FINANCE_DOMAIN', '金融领域知识', 'GENERAL', 6), +(507, 5, 'TOOLS', '工程化工具', 'TOOL', 7), +(601, 6, 'LANGUAGE', '编程语言', 'PROGRAMMING_LANGUAGE', 1), +(602, 6, 'STATISTICS_MODELING', '统计与建模', 'FRAMEWORK', 2), +(603, 6, 'RISK_MODEL', '风险模型', 'GENERAL', 3), +(604, 6, 'MODEL_PLATFORM', '建模框架/平台', 'FRAMEWORK', 4), +(605, 6, 'DATA_DATABASE', '数据处理与数据库', 'TOOL', 5), +(606, 6, 'MODEL_VALIDATION', '模型验证', 'GENERAL', 6), +(607, 6, 'DEPLOY_TOOLS', '部署与工程化工具', 'TOOL', 7), +(701, 7, 'LANGUAGE', '编程语言', 'PROGRAMMING_LANGUAGE', 1), +(702, 7, 'QUANT_PLATFORM', '量化建模框架/平台', 'FRAMEWORK', 2), +(703, 7, 'MARKET_DATA', '市场与金融数据', 'TOOL', 3), +(704, 7, 'DATABASE', '数据库', 'TOOL', 4), +(705, 7, 'PERFORMANCE', '低延迟与高性能工程', 'TOOL', 5), +(706, 7, 'TEST', '测试', 'GENERAL', 6), +(707, 7, 'CLOUD_DEVOPS', '云/DevOps', 'TOOL', 7), +(801, 8, 'EDU_DOMAIN', '教育领域知识', 'GENERAL', 1), +(802, 8, 'PRODUCT_ANALYSIS', '产品分析方法', 'TOOL', 2), +(803, 8, 'REQUIREMENTS_PROCESS', '需求与流程', 'GENERAL', 3), +(804, 8, 'DATA_ANALYSIS', '数据分析', 'TOOL', 4), +(805, 8, 'EDU_PLATFORM', '教育平台与工具', 'TOOL', 5), +(806, 8, 'COLLAB_TOOLS', '协作与工程化工具', 'GENERAL', 6), +(901, 9, 'INSTRUCTIONAL_DESIGN', '教学设计', 'FRAMEWORK', 1), +(902, 9, 'COURSE_PLATFORM', '课程与学习平台', 'TOOL', 2), +(903, 9, 'DIGITAL_CONTENT', '数字化内容工具', 'TOOL', 3), +(904, 9, 'ASSESSMENT', '评估设计', 'GENERAL', 4), +(905, 9, 'LEARNING_ANALYSIS', '学习数据分析', 'TOOL', 5), +(906, 9, 'ACCESSIBILITY', '无障碍与规范', 'GENERAL', 6), +(1001, 10, 'LANGUAGE', '编程语言', 'PROGRAMMING_LANGUAGE', 1), +(1002, 10, 'ANALYSIS_VISUALIZATION', '数据分析与可视化', 'TOOL', 2), +(1003, 10, 'DATABASE_SQL', '数据库与 SQL', 'TOOL', 3), +(1004, 10, 'STATISTICS_MODELING', '统计与建模', 'FRAMEWORK', 4), +(1005, 10, 'EDU_DOMAIN', '教育领域知识', 'GENERAL', 5), +(1006, 10, 'REPORTING_TOOLS', '报表与工程化工具', 'TOOL', 6); +INSERT INTO skills (id, code, name_zh, name_en, source_type, source_code, source_version_id, sort_order) VALUES +(1, 'JAVA', 'Java', 'Java (computer programming)', 'ESCO', 'http://data.europa.eu/esco/skill/19a8293b-8e95-4de3-983f-77484079c389', 1, 1), +(2, 'PYTHON', 'Python', 'Python (computer programming)', 'ESCO', 'http://data.europa.eu/esco/skill/ccd0a1d9-afda-43d9-b901-96344886e14d', 1, 2), +(3, 'JAVASCRIPT', 'JavaScript', 'JavaScript', 'ESCO', 'http://data.europa.eu/esco/skill/3cd569a2-4f88-4c1e-9995-8dce8c5e51a7', 1, 3), +(4, 'TYPESCRIPT', 'TypeScript', 'TypeScript', 'ESCO', 'http://data.europa.eu/esco/skill/867137fb-ff1b-4ca3-99f3-cb6969aa2c68', 1, 4), +(5, 'SQL', 'SQL', 'SQL', 'ESCO', 'http://data.europa.eu/esco/skill/598de5b0-5b58-4ea7-8058-a4bc4d18c742', 1, 5), +(6, 'CSHARP', 'C#', 'C#', 'ESCO', 'http://data.europa.eu/esco/skill/4c016b68-4116-468c-9dc6-42710c239e4a', 1, 6), +(7, 'HTML', 'HTML', 'use markup languages', 'ESCO', 'http://data.europa.eu/esco/skill/0af062de-eb43-41e9-9b96-249e2cd22d26', 1, 7), +(8, 'CSS', 'CSS', 'CSS', 'ESCO', 'http://data.europa.eu/esco/skill/e5d1f825-60ed-4bdd-872a-e748c387f777', 1, 8), +(9, 'R', 'R', 'R', 'ESCO', 'http://data.europa.eu/esco/skill/51586df8-1c46-4b47-8583-773cb63bf00b', 1, 9), +(10, 'MYSQL', 'MySQL', 'MySQL', 'ESCO', 'http://data.europa.eu/esco/skill/4da171e5-779c-4983-a76f-91c16751e99f', 1, 10), +(11, 'POSTGRESQL', 'PostgreSQL', 'PostgreSQL', 'ESCO', 'http://data.europa.eu/esco/skill/a8d07b5a-c1a1-42c6-9d53-db9c7a2ca996', 1, 11), +(12, 'STATISTICS', '统计学', 'statistics', 'ESCO', 'http://data.europa.eu/esco/skill/7ee4c2ea-b349-4bd2-81a3-ec31475d4833', 1, 12), +(13, 'MACHINE_LEARNING', '机器学习', 'utilise machine learning', 'ESCO', 'http://data.europa.eu/esco/skill/8369c2d6-c100-4cf6-bd83-9668d8678433', 1, 13), +(14, 'FINANCIAL_ANALYSIS', '金融分析', 'financial analysis', 'ESCO', 'http://data.europa.eu/esco/skill/99571e68-801f-49af-a897-5f75996642e1', 1, 14), +(15, 'RISK_MANAGEMENT', '风险管理', 'risk management', 'ESCO', 'http://data.europa.eu/esco/skill/6eff134b-e34f-4d6e-a6e8-5e47cf2228d0', 1, 15), +(16, 'INSTRUCTIONAL_DESIGN', '教学设计', 'instructional design models', 'ESCO', 'http://data.europa.eu/esco/skill/e3145bd7-7f35-44ec-a5dc-4417c5c9ab9a', 1, 16), +(17, 'CURRICULUM_DEVELOPMENT', '课程开发', 'advise on curriculum development', 'ESCO', 'http://data.europa.eu/esco/skill/142a5319-60cb-42a2-891f-90ba92af18b1', 1, 17), +(18, 'LEARNING_MANAGEMENT_SYSTEMS', '学习管理系统', 'learning management systems', 'ESCO', 'http://data.europa.eu/esco/skill/6c80d53c-d8c9-41fe-998f-091fca208834', 1, 18), +(19, 'EDUCATIONAL_TECHNOLOGY', '教育技术', 'develop e-learning plan', 'ESCO', 'http://data.europa.eu/esco/skill/8a6394d1-f44f-4b0d-81d5-7201fc025ba8', 1, 19), +(20, 'COMMUNICATION', '沟通能力', 'communication', 'ESCO', 'http://data.europa.eu/esco/skill/15d76317-c71a-4fa2-aadc-2ecc34e627b7', 1, 20), +(21, 'CRITICAL_THINKING', '批判性思维', 'use logical reasoning', 'ESCO', 'http://data.europa.eu/esco/skill/a9e6718b-b9d8-44c2-b06e-85ceed9d144c', 1, 21), +(22, 'COLLABORATION', '团队协作', 'cooperate with colleagues', 'ESCO', 'http://data.europa.eu/esco/skill/082967d4-b94c-4305-aa13-35a6aa8c5c7c', 1, 22), +(23, 'PROJECT_MANAGEMENT', '项目管理', 'project management', 'ESCO', 'http://data.europa.eu/esco/skill/7111b95d-0ce3-441a-9d92-4c75d05c4388', 1, 23), +(24, 'AGILE', '敏捷开发', 'Agile development', 'ESCO', 'http://data.europa.eu/esco/skill/dba46f87-0831-49cd-a1c7-340a653c0221', 1, 24), +(25, 'JENKINS', 'Jenkins', 'Jenkins (tools for software configuration management)', 'ESCO', 'http://data.europa.eu/esco/skill/f47a1998-0beb-43be-9f46-380aa4d183da', 1, 25), +(26, 'REACT', 'React', 'React', 'O*NET', '15-1254.00|React|2.E.7.h', 2, 26), +(27, 'VUE_JS', 'Vue.js', 'Vue.js', 'O*NET', '15-1254.00|Vue.js|2.E.7.h', 2, 27), +(28, 'NODE_JS', 'Node.js', 'Node.js', 'O*NET', '15-1254.00|Node.js|2.E.7.h', 2, 28), +(29, 'SPRING_BOOT', 'Spring Boot', 'Spring Boot', 'O*NET', '15-1252.00|Spring Boot|2.E.10.a', 2, 29), +(30, 'DJANGO', 'Django', 'Django', 'O*NET', '15-1252.00|Django|2.E.7.h', 2, 30), +(31, 'FLASK', 'Flask', 'Flask', 'O*NET', '15-2051.00|Flask|2.E.7.b', 2, 31), +(32, 'GIT', 'Git', 'Git', 'O*NET', '15-1252.00|Git|2.E.5.c', 2, 32), +(33, 'DOCKER', 'Docker', 'Docker', 'O*NET', '15-1252.00|Docker|2.E.10.a', 2, 33), +(34, 'KUBERNETES', 'Kubernetes', 'Kubernetes', 'O*NET', '15-1252.00|Kubernetes|2.E.10.a', 2, 34), +(35, 'GRADLE', 'Gradle', 'Gradle', 'O*NET', '15-1252.00|Gradle|2.E.7.e', 2, 35), +(36, 'MONGODB', 'MongoDB', 'MongoDB', 'O*NET', '15-1252.00|MongoDB|2.E.6.d', 2, 36), +(37, 'REDIS', 'Redis', 'Redis', 'O*NET', '15-1252.00|Redis|2.E.6.d', 2, 37), +(38, 'SELENIUM', 'Selenium', 'Selenium', 'O*NET', '15-1253.00|Selenium|2.E.7.f', 2, 38), +(39, 'JUNIT', 'JUnit', 'JUnit', 'O*NET', '15-1253.00|JUnit|2.E.7.f', 2, 39), +(40, 'POSTMAN', 'Postman', 'Postman', 'O*NET', '15-1253.00|Postman|2.E.13.c', 2, 40), +(41, 'PANDAS', 'pandas', 'pandas', 'O*NET', '15-2051.00|pandas|2.E.6.f', 2, 41), +(42, 'NUMPY', 'NumPy', 'NumPy', 'O*NET', '15-2051.00|NumPy|2.E.6.f', 2, 42), +(43, 'TABLEAU', 'Tableau', 'Tableau', 'O*NET', '15-2051.00|Tableau|2.E.6.m', 2, 43), +(44, 'POWER_BI', 'Microsoft Power BI', 'Microsoft Power BI', 'O*NET', '15-2051.00|Microsoft Power BI|2.E.6.m', 2, 44), +(45, 'MICROSOFT_EXCEL', 'Microsoft Excel', 'Microsoft Excel', 'O*NET', '15-2051.00|Microsoft Excel|2.E.4.i', 2, 45), +(46, 'TENSORFLOW', 'TensorFlow', 'TensorFlow', 'O*NET', '15-2051.00|TensorFlow|2.E.9.d', 2, 46), +(47, 'PYTORCH', 'PyTorch', 'PyTorch', 'O*NET', '15-2051.00|PyTorch|2.E.6.f', 2, 47), +(48, 'SCIKIT_LEARN', 'Scikit-learn', 'Scikit-learn', 'O*NET', '15-2051.00|Scikit-learn|2.E.7.b', 2, 48), +(49, 'APACHE_SPARK', 'Apache Spark', 'Apache Spark', 'O*NET', '15-2051.00|Apache Spark|2.E.6.m', 2, 49), +(50, 'FIGMA', 'Figma', 'Figma', 'O*NET', '15-1254.00|Figma|2.E.7.d', 2, 50), +(51, 'BASH', 'Bash', 'Bash', 'O*NET', '15-1252.00|Bash|2.E.13.c', 2, 51), +(52, 'GO', 'Go', 'Go', 'O*NET', '15-1252.00|Go|2.E.7.b', 2, 52), +(53, 'CPP', 'C++', 'C++', 'O*NET', '15-1252.00|C++|2.E.7.e', 2, 53), +(54, 'GOOGLE_CHROME', 'Google Chrome', 'Google Chrome', 'O*NET', '17-2051.02|Google Chrome|2.E.10.e', 2, 54), +(55, 'DEBUGGING_SOFTWARE', '调试软件', 'Debugging software', 'O*NET', '17-2071.00|Debugging software|2.E.7.f', 2, 55), +(56, 'APACHE_KAFKA', 'Apache Kafka', 'Apache Kafka', 'O*NET', '15-1299.07|Apache Kafka|2.E.7.b', 2, 56), +(57, 'SONARQUBE', 'SonarQube', 'SonarQube', 'O*NET', '15-1252.00|SonarQube|2.E.7.f', 2, 57), +(58, 'CHEF', 'Chef', 'Chef', 'O*NET', '15-1255.00|Chef|2.E.7.a', 2, 58), +(59, 'GITLAB', 'GitLab', 'GitLab', 'O*NET', '15-1299.08|GitLab|2.E.10.a', 2, 59), +(60, 'NAGIOS', 'Nagios', 'Nagios', 'O*NET', '11-3021.00|Nagios|2.E.11.a', 2, 60), +(61, 'NMAP', 'Nmap', 'Nmap', 'O*NET', '13-1199.07|Nmap|2.E.14.d', 2, 61), +(62, 'BLOOMBERG_PROFESSIONAL', 'Bloomberg Professional', 'Bloomberg Professional', 'O*NET', '41-3031.00|Bloomberg Professional|2.E.2.d', 2, 62), +(63, 'MOODLE', 'Moodle', 'Moodle', 'O*NET', '25-2022.00|Moodle|2.E.8.b', 2, 63), +(64, 'ARTICULATE_STORYLINE', 'Articulate Storyline', 'Articulate Storyline', 'O*NET', '25-9031.00|Articulate Storyline|2.E.8.b', 2, 64), +(65, 'MICROSOFT_POWERPOINT', 'Microsoft PowerPoint', 'Microsoft PowerPoint', 'O*NET', '27-4012.00|Microsoft PowerPoint|2.E.4.f', 2, 65), +(66, 'AXURE_RP', 'Axure RP', 'Axure RP', 'O*NET', '15-1255.00|Axure RP|2.E.4.g', 2, 66), +(67, 'ADOBE_PHOTOSHOP', 'Adobe Photoshop', 'Adobe Photoshop', 'O*NET', '19-3092.00|Adobe Photoshop|2.E.4.b', 2, 67), +(68, 'APACHE_JMETER', 'Apache JMeter', 'Apache JMeter', 'O*NET', '15-1253.00|Apache JMeter|2.E.7.e', 2, 68), +(69, 'ADOBE_ACROBAT', 'Adobe Acrobat', 'Adobe Acrobat', 'O*NET', '11-1011.00|Adobe Acrobat|2.E.5.b', 2, 69), +(70, 'MICROSOFT_AZURE', 'Microsoft Azure', 'Microsoft Azure software', 'O*NET', '41-4011.00|Microsoft Azure software|2.E.7.b', 2, 70), +(71, 'AWS', 'AWS', 'Amazon Web Services AWS software', 'O*NET', '11-1021.00|Amazon Web Services AWS software|2.E.6.f', 2, 71), +(72, 'WIRESHARK', 'Wireshark', 'Wireshark', 'O*NET', '15-1241.01|Wireshark|2.E.11.a', 2, 72); +INSERT INTO skill_aliases (id, skill_id, alias, normalized_alias) VALUES +(1, 1, 'Java (computer programming)', 'java (computer programming)'), +(2, 2, 'Python (computer programming)', 'python (computer programming)'), +(3, 3, 'JavaScript', 'javascript'), +(4, 4, 'TypeScript', 'typescript'), +(5, 5, 'SQL', 'sql'), +(6, 6, 'C#', 'c#'), +(7, 7, 'HTML', 'html'), +(8, 8, 'CSS', 'css'), +(9, 9, 'R', 'r'), +(10, 10, 'MySQL', 'mysql'), +(11, 11, 'PostgreSQL', 'postgresql'), +(12, 12, 'statistics', 'statistics'), +(13, 13, 'machine learning', 'machine learning'), +(14, 14, 'financial analysis', 'financial analysis'), +(15, 15, 'risk management', 'risk management'), +(16, 16, 'instructional design', 'instructional design'), +(17, 17, 'curriculum development', 'curriculum development'), +(18, 18, 'learning management systems', 'learning management systems'), +(19, 19, 'educational technology', 'educational technology'), +(20, 20, 'communication', 'communication'), +(21, 21, 'critical thinking', 'critical thinking'), +(22, 22, 'collaboration', 'collaboration'), +(23, 23, 'project management', 'project management'), +(24, 24, 'agile', 'agile'), +(25, 25, 'Jenkins (tools for software configuration management)', 'jenkins (tools for software configuration management)'), +(26, 26, 'React', 'react'), +(27, 27, 'Vue.js', 'vuejs'), +(28, 28, 'Node.js', 'nodejs'), +(29, 29, 'Spring Boot', 'spring boot'), +(30, 30, 'Django', 'django'), +(31, 31, 'Flask', 'flask'), +(32, 32, 'Git', 'git'), +(33, 33, 'Docker', 'docker'), +(34, 34, 'Kubernetes', 'kubernetes'), +(35, 35, 'Gradle', 'gradle'), +(36, 36, 'MongoDB', 'mongodb'), +(37, 37, 'Redis', 'redis'), +(38, 38, 'Selenium', 'selenium'), +(39, 39, 'JUnit', 'junit'), +(40, 40, 'Postman', 'postman'), +(41, 41, 'pandas', 'pandas'), +(42, 42, 'NumPy', 'numpy'), +(43, 43, 'Tableau', 'tableau'), +(44, 44, 'Microsoft Power BI', 'microsoft power bi'), +(45, 45, 'Microsoft Excel', 'microsoft excel'), +(46, 46, 'TensorFlow', 'tensorflow'), +(47, 47, 'PyTorch', 'pytorch'), +(48, 48, 'scikit-learn', 'scikit learn'), +(49, 49, 'Apache Spark', 'apache spark'), +(50, 50, 'Figma', 'figma'), +(51, 51, 'Bash', 'bash'), +(52, 52, 'Go', 'go'), +(53, 53, 'C++', 'c++'), +(54, 54, 'Google Chrome', 'google chrome'), +(55, 55, 'debugging software', 'debugging software'), +(56, 56, 'Apache Kafka', 'apache kafka'), +(57, 57, 'SonarQube', 'sonarqube'), +(58, 58, 'Chef', 'chef'), +(59, 59, 'GitLab', 'gitlab'), +(60, 60, 'Nagios', 'nagios'), +(61, 61, 'Nmap', 'nmap'), +(62, 62, 'Bloomberg Professional', 'bloomberg professional'), +(63, 63, 'Moodle', 'moodle'), +(64, 64, 'Articulate Storyline', 'articulate storyline'), +(65, 65, 'Microsoft PowerPoint', 'microsoft powerpoint'), +(66, 66, 'Axure RP', 'axure rp'), +(67, 67, 'Adobe Photoshop', 'adobe photoshop'), +(68, 68, 'Apache JMeter', 'apache jmeter'), +(69, 69, 'Adobe Acrobat', 'adobe acrobat'), +(70, 70, 'Microsoft Azure software', 'microsoft azure software'), +(71, 71, 'Amazon Web Services AWS software', 'amazon web services aws software'), +(72, 72, 'Wireshark', 'wireshark'); +INSERT INTO occupation_skills (id, occupation_id, skill_id, category_id, importance, importance_source_code, importance_element_id, importance_is_proxy, relationship_type, source_version_id) VALUES +(1, 1, 3, 101, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(2, 1, 4, 101, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(3, 1, 7, 102, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(4, 1, 8, 102, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(5, 1, 26, 103, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(6, 1, 27, 103, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(7, 1, 28, 104, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(8, 1, 35, 104, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(9, 1, 54, 105, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(10, 1, 55, 105, 4.88, '15-1254.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(11, 1, 38, 106, 3.33, '15-1254.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(12, 1, 39, 106, 3.33, '15-1254.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(13, 1, 32, 107, 3.88, '15-1254.00', '4.A.4.a.2', 1, 'ESSENTIAL', 2), +(14, 1, 24, 107, 3.88, '15-1254.00', '4.A.4.a.2', 1, 'ESSENTIAL', 2), +(15, 2, 1, 201, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(16, 2, 2, 201, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(17, 2, 29, 202, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(18, 2, 30, 202, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(19, 2, 10, 203, 4.38, '15-1252.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(20, 2, 11, 203, 4.38, '15-1252.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(21, 2, 37, 204, 4.38, '15-1252.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(22, 2, 56, 204, 4.38, '15-1252.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(23, 2, 33, 205, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(24, 2, 34, 205, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(25, 2, 39, 206, 3.94, '15-1252.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(26, 2, 40, 206, 3.94, '15-1252.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(27, 2, 32, 207, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(28, 2, 35, 207, 4.61, '15-1252.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(29, 2, 61, 208, 2.57, '15-1252.00', '2.C.8.a', 1, 'ESSENTIAL', 2), +(30, 2, 57, 208, 2.57, '15-1252.00', '2.C.8.a', 1, 'ESSENTIAL', 2), +(31, 3, 1, 301, 4.94, '15-1253.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(32, 3, 2, 301, 4.94, '15-1253.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(33, 3, 40, 302, 4.24, '15-1253.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(34, 3, 21, 302, 4.24, '15-1253.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(35, 3, 38, 303, 4.94, '15-1253.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(36, 3, 39, 303, 4.94, '15-1253.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(37, 3, 5, 304, 4.38, '15-1253.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(38, 3, 10, 304, 4.38, '15-1253.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(39, 3, 68, 305, 2.94, '15-1253.00', '4.A.1.b.3', 1, 'ESSENTIAL', 2), +(40, 3, 51, 305, 2.94, '15-1253.00', '4.A.1.b.3', 1, 'ESSENTIAL', 2), +(41, 3, 25, 306, 4.94, '15-1253.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(42, 3, 59, 306, 4.94, '15-1253.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(43, 4, 2, 401, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(44, 4, 51, 401, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(45, 4, 71, 402, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(46, 4, 70, 402, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(47, 4, 33, 403, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(48, 4, 34, 403, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(49, 4, 25, 404, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(50, 4, 59, 404, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(51, 4, 58, 405, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(52, 4, 32, 405, 4.83, '15-1244.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(53, 4, 60, 406, 3.60, '15-1244.00', '4.A.1.a.2', 1, 'ESSENTIAL', 2), +(54, 4, 72, 406, 3.60, '15-1244.00', '4.A.1.a.2', 1, 'ESSENTIAL', 2), +(55, 4, 61, 407, 2.34, '15-1244.00', '2.C.8.a', 1, 'ESSENTIAL', 2), +(56, 4, 57, 407, 2.34, '15-1244.00', '2.C.8.a', 1, 'ESSENTIAL', 2), +(57, 4, 10, 408, 4.03, '15-1244.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(58, 4, 11, 408, 4.03, '15-1244.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(59, 5, 2, 501, 4.48, '13-2031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(60, 5, 9, 501, 4.48, '13-2031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(61, 5, 41, 502, 4.01, '13-2031.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(62, 5, 43, 502, 4.01, '13-2031.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(63, 5, 5, 503, 4.44, '13-2031.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(64, 5, 11, 503, 4.44, '13-2031.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(65, 5, 12, 504, 4.03, '13-2031.00', '2.C.4.a', 1, 'ESSENTIAL', 2), +(66, 5, 13, 504, 4.03, '13-2031.00', '2.C.4.a', 1, 'ESSENTIAL', 2), +(67, 5, 62, 505, 4.48, '13-2031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(68, 5, 45, 505, 4.48, '13-2031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(69, 5, 14, 506, 4.42, '13-2031.00', '2.C.1.c', 1, 'ESSENTIAL', 2), +(70, 5, 15, 506, 4.42, '13-2031.00', '2.C.1.c', 1, 'ESSENTIAL', 2), +(71, 5, 32, 507, 4.48, '13-2031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(72, 5, 44, 507, 4.48, '13-2031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(73, 6, 2, 601, 4.41, '13-2053.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(74, 6, 9, 601, 4.41, '13-2053.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(75, 6, 12, 602, 3.27, '13-2053.00', '2.C.4.a', 1, 'ESSENTIAL', 2), +(76, 6, 13, 602, 3.27, '13-2053.00', '2.C.4.a', 1, 'ESSENTIAL', 2), +(77, 6, 15, 603, 4.43, '13-2053.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(78, 6, 21, 603, 4.43, '13-2053.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(79, 6, 46, 604, 4.41, '13-2053.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(80, 6, 48, 604, 4.41, '13-2053.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(81, 6, 5, 605, 4.13, '13-2053.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(82, 6, 49, 605, 4.13, '13-2053.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(83, 6, 39, 606, 3.90, '13-2053.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(84, 6, 57, 606, 3.90, '13-2053.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(85, 6, 33, 607, 4.41, '13-2053.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(86, 6, 32, 607, 4.41, '13-2053.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(87, 7, 2, 701, 4.68, '13-2099.01', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(88, 7, 53, 701, 4.68, '13-2099.01', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(89, 7, 52, 701, 4.68, '13-2099.01', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(90, 7, 42, 702, 4.68, '13-2099.01', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(91, 7, 48, 702, 4.68, '13-2099.01', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(92, 7, 62, 703, 4.20, '13-2099.01', '2.C.1.c', 1, 'ESSENTIAL', 2), +(93, 7, 14, 703, 4.20, '13-2099.01', '2.C.1.c', 1, 'ESSENTIAL', 2), +(94, 7, 11, 704, 4.50, '13-2099.01', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(95, 7, 5, 704, 4.50, '13-2099.01', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(96, 7, 49, 705, 3.53, '13-2099.01', '4.A.1.b.3', 1, 'ESSENTIAL', 2), +(97, 7, 56, 705, 3.53, '13-2099.01', '4.A.1.b.3', 1, 'ESSENTIAL', 2), +(98, 7, 39, 706, 3.05, '13-2099.01', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(99, 7, 68, 706, 3.05, '13-2099.01', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(100, 7, 33, 707, 4.68, '13-2099.01', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(101, 7, 34, 707, 4.68, '13-2099.01', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(102, 8, 16, 801, 2.52, '13-1082.00', '2.C.6', 1, 'ESSENTIAL', 2), +(103, 8, 17, 801, 2.52, '13-1082.00', '2.C.6', 1, 'ESSENTIAL', 2), +(104, 8, 66, 802, 3.94, '13-1082.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(105, 8, 44, 802, 3.94, '13-1082.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(106, 8, 23, 803, 3.73, '13-1082.00', '4.A.2.b.4', 1, 'ESSENTIAL', 2), +(107, 8, 24, 803, 3.73, '13-1082.00', '4.A.2.b.4', 1, 'ESSENTIAL', 2), +(108, 8, 43, 804, 3.94, '13-1082.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(109, 8, 2, 804, 3.94, '13-1082.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(110, 8, 18, 805, 4.45, '13-1082.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(111, 8, 63, 805, 4.45, '13-1082.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(112, 8, 32, 806, 4.73, '13-1082.00', '4.A.4.a.2', 1, 'ESSENTIAL', 2), +(113, 8, 45, 806, 4.73, '13-1082.00', '4.A.4.a.2', 1, 'ESSENTIAL', 2), +(114, 9, 16, 901, 4.78, '25-9031.00', '2.C.6', 1, 'ESSENTIAL', 2), +(115, 9, 17, 901, 4.78, '25-9031.00', '2.C.6', 1, 'ESSENTIAL', 2), +(116, 9, 18, 902, 4.45, '25-9031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(117, 9, 63, 902, 4.45, '25-9031.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(118, 9, 64, 903, 3.72, '25-9031.00', '4.A.2.b.2', 1, 'ESSENTIAL', 2), +(119, 9, 50, 903, 3.72, '25-9031.00', '4.A.2.b.2', 1, 'ESSENTIAL', 2), +(120, 9, 19, 904, 3.83, '25-9031.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(121, 9, 21, 904, 3.83, '25-9031.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(122, 9, 12, 905, 4.19, '25-9031.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(123, 9, 45, 905, 4.19, '25-9031.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(124, 9, 69, 906, 3.83, '25-9031.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(125, 9, 20, 906, 3.83, '25-9031.00', '4.A.2.a.3', 1, 'ESSENTIAL', 2), +(126, 10, 2, 1001, 4.75, '15-2051.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(127, 10, 9, 1001, 4.75, '15-2051.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(128, 10, 43, 1002, 4.83, '15-2051.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(129, 10, 44, 1002, 4.83, '15-2051.00', '4.A.2.a.4', 1, 'ESSENTIAL', 2), +(130, 10, 5, 1003, 4.71, '15-2051.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(131, 10, 11, 1003, 4.71, '15-2051.00', '4.A.2.a.2', 1, 'ESSENTIAL', 2), +(132, 10, 12, 1004, 4.52, '15-2051.00', '2.C.4.a', 1, 'ESSENTIAL', 2), +(133, 10, 13, 1004, 4.52, '15-2051.00', '2.C.4.a', 1, 'ESSENTIAL', 2), +(134, 10, 17, 1005, 2.28, '15-2051.00', '2.C.6', 1, 'ESSENTIAL', 2), +(135, 10, 19, 1005, 2.28, '15-2051.00', '2.C.6', 1, 'ESSENTIAL', 2), +(136, 10, 45, 1006, 4.75, '15-2051.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(137, 10, 65, 1006, 4.75, '15-2051.00', '4.A.3.b.1', 1, 'ESSENTIAL', 2), +(138, 1, 50, 104, NULL, NULL, NULL, 0, 'OPTIONAL', 2), +(139, 9, 67, 903, NULL, NULL, NULL, 0, 'OPTIONAL', 2); diff --git a/mydreamjob-server/src/main/resources/db/migration/V37__skill_cloud_boolean_type.sql b/mydreamjob-server/src/main/resources/db/migration/V37__skill_cloud_boolean_type.sql new file mode 100644 index 0000000..6fdc95d --- /dev/null +++ b/mydreamjob-server/src/main/resources/db/migration/V37__skill_cloud_boolean_type.sql @@ -0,0 +1,12 @@ +-- Flyway V37: 将技能代理标记规范化为跨方言 BOOLEAN。 +-- 不改写 V35/V36，确保已应用迁移的 Flyway checksum 保持有效。 + +ALTER TABLE occupation_skills + DROP CONSTRAINT ck_occupation_skills_importance_proxy; + +ALTER TABLE occupation_skills + MODIFY COLUMN importance_is_proxy BOOLEAN NOT NULL DEFAULT FALSE; + +ALTER TABLE occupation_skills + ADD CONSTRAINT ck_occupation_skills_importance_proxy + CHECK (importance_is_proxy IN (FALSE, TRUE)); \ No newline at end of file diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudAuthorizationIntegrationTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudAuthorizationIntegrationTest.java new file mode 100644 index 0000000..1678bf7 --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudAuthorizationIntegrationTest.java @@ -0,0 +1,236 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.auth.JwtTokenProvider; +import com.mydreamjob.auth.User; +import com.mydreamjob.auth.UserRepository; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillCloudNode; +import com.mydreamjob.skill.dto.SkillCloudResponse; +import com.mydreamjob.skill.dto.SkillLearningPathItem; +import com.mydreamjob.skill.dto.SkillLearningPathResponse; +import com.mydreamjob.skill.dto.SkillMatchResponse; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.data.redis.core.ValueOperations; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; +import org.springframework.test.web.servlet.MockMvc; + +import java.math.BigDecimal; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.atomic.AtomicLong; + +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.isNull; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; + +@SpringBootTest +@AutoConfigureMockMvc +@ActiveProfiles("integration-test") +@TestPropertySource(properties = + "spring.datasource.url=jdbc:h2:mem:skill_controller_auth;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE") +class SkillCloudAuthorizationIntegrationTest { + + private static final String WORD_CLOUD_PATH = "/skills/word-cloud"; + private static final String MATCH_PATH = "/skills/match"; + private static final String LEARNING_PATH = "/skills/learning-path"; + private static final List PATHS = List.of(WORD_CLOUD_PATH, MATCH_PATH, LEARNING_PATH); + private static final AtomicLong PHONE = new AtomicLong(920_000_000L); + + @Autowired + private MockMvc mockMvc; + + @Autowired + private JwtTokenProvider jwtTokenProvider; + + @Autowired + private UserRepository userRepository; + + @MockBean + private SkillCloudQueryService skillCloudQueryService; + + @MockBean + private SkillMatchService skillMatchService; + + @MockBean + private SkillLearningPathService skillLearningPathService; + + @MockBean + private RedisTemplate redisTemplate; + + @BeforeEach + @SuppressWarnings("unchecked") + void setUp() { + ValueOperations operations = mock(ValueOperations.class); + when(redisTemplate.opsForValue()).thenReturn(operations); + when(redisTemplate.hasKey(any())).thenReturn(false); + + when(skillCloudQueryService.query(1L, 1L, null)).thenReturn(cloudResponse()); + when(skillMatchService.match(anyLong(), eq(1L), eq(1L), isNull())).thenReturn(matchResponse()); + when(skillLearningPathService.generate(anyLong(), eq(1L), eq(1L), isNull())) + .thenReturn(learningPathResponse()); + } + + @AfterEach + void tearDown() { + userRepository.deleteAll(); + } + + @Test + void databaseRoleSetsControlAllSkillEndpoints() throws Exception { + Long student = createUser(Set.of(User.Role.STUDENT)); + Long company = createUser(Set.of(User.Role.COMPANY)); + Long admin = createUser(Set.of(User.Role.ADMIN)); + Long studentCompany = createUser(Set.of(User.Role.STUDENT, User.Role.COMPANY)); + Long companyAdmin = createUser(Set.of(User.Role.COMPANY, User.Role.ADMIN)); + + for (String path : PATHS) { + assertStatus(path, student, 200); + assertStatus(path, company, 403); + assertStatus(path, admin, 403); + assertStatus(path, studentCompany, 200); + assertStatus(path, companyAdmin, 403); + } + } + + @Test + void anonymousAndInvalidJwtUseSecurityEntryPointContract() throws Exception { + for (String path : PATHS) { + assertUnauthorized(path, null); + assertUnauthorized(path, "Bearer invalid-token"); + } + } + + @Test + void identityQueryParametersCannotOverrideAuthenticatedSubject() throws Exception { + Long student = createUser(Set.of(User.Role.STUDENT)); + Long company = createUser(Set.of(User.Role.COMPANY)); + + for (String path : PATHS) { + mockMvc.perform(get(path) + .param("industryId", "1") + .param("occupationId", "1") + .param("userId", company.toString()) + .param("studentId", company.toString()) + .header("Authorization", bearerToken(student))) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.code").value(200)); + } + + verify(skillCloudQueryService).query(1L, 1L, null); + verify(skillMatchService).match(student, 1L, 1L, null); + verify(skillLearningPathService).generate(student, 1L, 1L, null); + } + + private void assertUnauthorized(String path, String authorization) throws Exception { + var request = get(path) + .param("industryId", "1") + .param("occupationId", "1"); + if (authorization != null) { + request.header("Authorization", authorization); + } + mockMvc.perform(request) + .andExpect(status().isUnauthorized()) + .andExpect(jsonPath("$.code").value(401)) + .andExpect(jsonPath("$.message").value("not authenticated")) + .andExpect(jsonPath("$.timestamp").value("")) + .andExpect(jsonPath("$.requestId").doesNotExist()) + .andExpect(jsonPath("$.data").doesNotExist()); + } + + private void assertStatus(String path, Long userId, int expectedStatus) throws Exception { + var request = get(path) + .param("industryId", "1") + .param("occupationId", "1") + .header("Authorization", bearerToken(userId)); + if (expectedStatus == 200) { + mockMvc.perform(request) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.code").value(200)); + } else { + mockMvc.perform(request) + .andExpect(status().isForbidden()) + .andExpect(jsonPath("$.code").value(2002)); + } + } + + private Long createUser(Set roles) { + User user = User.builder() + .phone(String.valueOf(PHONE.getAndIncrement())) + .role(roles.iterator().next()) + .roles(Set.copyOf(roles)) + .active(true) + .verified(true) + .build(); + return userRepository.save(user).getId(); + } + + private String bearerToken(Long userId) { + return "Bearer " + jwtTokenProvider.generateAccessToken( + userId, + String.valueOf(userId), + Map.of("roles", "STUDENT") + ); + } + + private SkillCloudResponse cloudResponse() { + return new SkillCloudResponse( + new SkillCloudNode(1L, "INTERNET", "互联网"), + new SkillCloudNode(1L, "FRONTEND_DEVELOPER", "前端开发工程师"), + null, + 1, + List.of(skill()) + ); + } + + private SkillMatchResponse matchResponse() { + return new SkillMatchResponse(100, 1, 1, List.of(skill()), List.of()); + } + + private SkillLearningPathResponse learningPathResponse() { + return new SkillLearningPathResponse( + 1, + List.of(new SkillLearningPathItem( + 1, + 3L, + "JavaScript", + 101L, + "编程语言", + "先掌握 JavaScript 的基础语法和常见数据结构，再完成至少一个编码练习。" + )) + ); + } + + private SkillCloudItem skill() { + return new SkillCloudItem( + 1, + 3L, + "JAVASCRIPT", + "JavaScript", + "JavaScript", + "ESCO", + "http://data.europa.eu/esco/skill/javascript", + 101L, + "编程语言", + new BigDecimal("4.38"), + "15-1254.00", + "2.B.3.e", + false + ); + } +} \ No newline at end of file diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudControllerTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudControllerTest.java new file mode 100644 index 0000000..032189b --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudControllerTest.java @@ -0,0 +1,430 @@ +package com.mydreamjob.skill; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.mydreamjob.auth.User; +import com.mydreamjob.auth.UserRepository; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillCloudNode; +import com.mydreamjob.skill.dto.SkillCloudResponse; +import com.mydreamjob.skill.dto.SkillLearningPathItem; +import com.mydreamjob.skill.dto.SkillLearningPathResponse; +import com.mydreamjob.skill.dto.SkillMatchResponse; +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.ValueSource; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.data.redis.core.ValueOperations; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; +import org.springframework.test.web.servlet.MockMvc; + +import java.math.BigDecimal; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.atomic.AtomicLong; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.isNull; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; + +@SpringBootTest +@AutoConfigureMockMvc +@ActiveProfiles("integration-test") +@TestPropertySource(properties = + "spring.datasource.url=jdbc:h2:mem:skill_controller_api;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE") +class SkillCloudControllerTest { + + private static final String WORD_CLOUD_PATH = "/skills/word-cloud"; + private static final String MATCH_PATH = "/skills/match"; + private static final String LEARNING_PATH = "/skills/learning-path"; + private static final AtomicLong PHONE = new AtomicLong(910_000_000L); + + private Long studentId; + + @Autowired + private MockMvc mockMvc; + + @Autowired + private ObjectMapper objectMapper; + + @Autowired + private com.mydreamjob.auth.JwtTokenProvider jwtTokenProvider; + + @Autowired + private UserRepository userRepository; + + @MockBean + private SkillCloudQueryService skillCloudQueryService; + + @MockBean + private SkillMatchService skillMatchService; + + @MockBean + private SkillLearningPathService skillLearningPathService; + + @MockBean + private RedisTemplate redisTemplate; + + @BeforeEach + @SuppressWarnings("unchecked") + void setUp() { + ValueOperations operations = mock(ValueOperations.class); + when(redisTemplate.opsForValue()).thenReturn(operations); + when(redisTemplate.hasKey(any())).thenReturn(false); + + studentId = userRepository.save(User.builder() + .phone(String.valueOf(PHONE.getAndIncrement())) + .role(User.Role.STUDENT) + .roles(Set.of(User.Role.STUDENT)) + .active(true) + .verified(true) + .build()).getId(); + + when(skillCloudQueryService.query(1L, 1L, null)).thenReturn(cloudResponse()); + when(skillMatchService.match(studentId, 1L, 1L, null)).thenReturn(matchResponse()); + when(skillLearningPathService.generate(studentId, 1L, 1L, null)).thenReturn(learningPathResponse()); + } + + @AfterEach + void tearDown() { + userRepository.deleteAll(); + } + + @Test + void wordCloudStudentRequestReturnsServicePayload() throws Exception { + mockMvc.perform(get(WORD_CLOUD_PATH) + .param("industryId", "1") + .param("occupationId", "1") + .header("Authorization", studentToken())) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.code").value(200)) + .andExpect(jsonPath("$.data.industry.code").value("INTERNET")) + .andExpect(jsonPath("$.data.occupation.code").value("FRONTEND_DEVELOPER")) + .andExpect(jsonPath("$.data.total").value(1)) + .andExpect(jsonPath("$.data.items[0].skillId").value(3)) + .andExpect(jsonPath("$.data.items[0].nameZh").value("JavaScript")) + .andExpect(jsonPath("$.data.items[0].sourceType").value("ESCO")); + } + + @Test + void matchStudentRequestReturnsServicePayload() throws Exception { + mockMvc.perform(get(MATCH_PATH) + .param("industryId", "1") + .param("occupationId", "1") + .header("Authorization", studentToken())) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.code").value(200)) + .andExpect(jsonPath("$.data.matchScore").value(100)) + .andExpect(jsonPath("$.data.matchedCount").value(1)) + .andExpect(jsonPath("$.data.totalCount").value(1)) + .andExpect(jsonPath("$.data.matchedSkills[0].skillId").value(3)) + .andExpect(jsonPath("$.data.missingSkills").isEmpty()); + } + + @Test + void learningPathStudentRequestReturnsServicePayload() throws Exception { + mockMvc.perform(get(LEARNING_PATH) + .param("industryId", "1") + .param("occupationId", "1") + .header("Authorization", studentToken())) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.code").value(200)) + .andExpect(jsonPath("$.data.total").value(1)) + .andExpect(jsonPath("$.data.items[0].priority").value(1)) + .andExpect(jsonPath("$.data.items[0].skillId").value(3)) + .andExpect(jsonPath("$.data.items[0].suggestion") + .value("先掌握 JavaScript 的基础语法和常见数据结构，再完成至少一个编码练习。")); + } + + @ParameterizedTest + @ValueSource(strings = {WORD_CLOUD_PATH, MATCH_PATH, LEARNING_PATH}) + void anonymousRequestUsesSecurityEntryPointContract(String path) throws Exception { + var result = mockMvc.perform(get(path) + .param("industryId", "1") + .param("occupationId", "1")) + .andExpect(status().isUnauthorized()) + .andExpect(jsonPath("$.code").value(401)) + .andExpect(jsonPath("$.message").value("not authenticated")) + .andExpect(jsonPath("$.timestamp").value("")) + .andExpect(jsonPath("$.requestId").doesNotExist()) + .andExpect(jsonPath("$.data").doesNotExist()) + .andReturn(); + + JsonNode body = objectMapper.readTree( + result.getResponse().getContentAsString(StandardCharsets.UTF_8)); + assertFields(body, Set.of("code", "message", "timestamp")); + } + + + @Test + void missingRequiredParametersReturnBadRequest() throws Exception { + mockMvc.perform(get(WORD_CLOUD_PATH) + .param("occupationId", "1") + .header("Authorization", studentToken())) + .andExpect(status().isBadRequest()) + .andExpect(jsonPath("$.code").value(400)); + + mockMvc.perform(get(WORD_CLOUD_PATH) + .param("industryId", "1") + .header("Authorization", studentToken())) + .andExpect(status().isBadRequest()) + .andExpect(jsonPath("$.code").value(400)); + } + + @Test + void nonPositiveParametersReturnBadRequest() throws Exception { + mockMvc.perform(get(WORD_CLOUD_PATH) + .param("industryId", "0") + .param("occupationId", "1") + .header("Authorization", studentToken())) + .andExpect(status().isBadRequest()) + .andExpect(jsonPath("$.code").value(400)); + + mockMvc.perform(get(MATCH_PATH) + .param("industryId", "1") + .param("occupationId", "-1") + .header("Authorization", studentToken())) + .andExpect(status().isBadRequest()) + .andExpect(jsonPath("$.code").value(400)); + + mockMvc.perform(get(LEARNING_PATH) + .param("industryId", "1") + .param("occupationId", "1") + .param("categoryId", "0") + .header("Authorization", studentToken())) + .andExpect(status().isBadRequest()) + .andExpect(jsonPath("$.code").value(400)); + } + + @Test + void identityComesOnlyFromAuthenticatedPrincipal() throws Exception { + mockMvc.perform(get(MATCH_PATH) + .param("industryId", "1") + .param("occupationId", "1") + .param("userId", "999") + .param("studentId", "888") + .header("Authorization", studentToken())) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.data.matchScore").value(100)); + + verify(skillMatchService).match(eq(studentId), eq(1L), eq(1L), isNull()); + } + + @ParameterizedTest + @ValueSource(strings = {WORD_CLOUD_PATH, MATCH_PATH, LEARNING_PATH}) + void successfulResponsesDoNotExposePrivateProfileFields(String path) throws Exception { + String body = mockMvc.perform(get(path) + .param("industryId", "1") + .param("occupationId", "1") + .header("Authorization", studentToken())) + .andExpect(status().isOk()) + .andReturn() + .getResponse() + .getContentAsString(StandardCharsets.UTF_8); + + assertThat(body) + .doesNotContain("skills_json") + .doesNotContain("phone") + .doesNotContain("email") + .doesNotContain("resume"); + } + + @Test + void skillOperationIdsAreExactAndGloballyUnique() throws Exception { + String json = mockMvc.perform(get("/v3/api-docs")) + .andExpect(status().isOk()) + .andReturn() + .getResponse() + .getContentAsString(StandardCharsets.UTF_8); + JsonNode paths = objectMapper.readTree(json).path("paths"); + + Map> locationsByOperationId = new HashMap<>(); + paths.fields().forEachRemaining(path -> path.getValue().fields().forEachRemaining(operation -> { + String operationId = operation.getValue().path("operationId").asText(); + if (!operationId.isBlank()) { + locationsByOperationId + .computeIfAbsent(operationId, ignored -> new ArrayList<>()) + .add(operation.getKey().toUpperCase(Locale.ROOT) + " " + path.getKey()); + } + })); + + Map expectedOperationIds = Map.of( + WORD_CLOUD_PATH, "getWordCloud", + MATCH_PATH, "getSkillMatch", + LEARNING_PATH, "getLearningPath"); + expectedOperationIds.forEach((path, operationId) -> { + assertThat(paths.path(path).path("get").path("operationId").asText()).isEqualTo(operationId); + assertThat(locationsByOperationId.getOrDefault(operationId, List.of())) + .containsExactly("GET " + path); + }); + } + + @Test + void openApiDocumentsStudentProtectedSkillEndpoints() throws Exception { + String json = mockMvc.perform(get("/v3/api-docs")) + .andExpect(status().isOk()) + .andReturn() + .getResponse() + .getContentAsString(StandardCharsets.UTF_8); + JsonNode root = objectMapper.readTree(json); + JsonNode paths = root.path("paths"); + + for (String path : List.of(WORD_CLOUD_PATH, MATCH_PATH, LEARNING_PATH)) { + JsonNode get = paths.path(path).path("get"); + assertThat(get.isMissingNode()).isFalse(); + assertThat(get.path("security").toString()).contains("bearerAuth"); + assertThat(get.path("responses").has("200")).isTrue(); + assertThat(get.path("responses").has("400")).isTrue(); + assertThat(get.path("responses").has("401")).isTrue(); + assertThat(get.path("responses").has("403")).isTrue(); + + JsonNode parameters = get.path("parameters"); + assertThat(parameter(parameters, "industryId").path("required").asBoolean()).isTrue(); + assertThat(parameter(parameters, "occupationId").path("required").asBoolean()).isTrue(); + assertThat(parameter(parameters, "categoryId").path("required").asBoolean()).isFalse(); + } + } + + @Test + void openApiSuccessExamplesMatchDtoContract() throws Exception { + String json = mockMvc.perform(get("/v3/api-docs")) + .andExpect(status().isOk()) + .andReturn() + .getResponse() + .getContentAsString(StandardCharsets.UTF_8); + JsonNode paths = objectMapper.readTree(json).path("paths"); + + JsonNode wordCloud = successExample(paths.path(WORD_CLOUD_PATH).path("get")); + JsonNode wordCloudData = wordCloud.path("data"); + assertFields(wordCloudData, Set.of("industry", "occupation", "category", "total", "items")); + assertThat(wordCloudData.path("total").asInt()).isEqualTo(wordCloudData.path("items").size()); + assertSkillItemContract(wordCloudData.path("items").get(0)); + + JsonNode match = successExample(paths.path(MATCH_PATH).path("get")); + JsonNode matchData = match.path("data"); + assertFields(matchData, Set.of("matchScore", "matchedCount", "totalCount", "matchedSkills", "missingSkills")); + int matchedCount = matchData.path("matchedCount").asInt(); + int totalCount = matchData.path("totalCount").asInt(); + assertThat(matchData.path("matchedSkills").size()).isEqualTo(matchedCount); + assertThat(matchData.path("missingSkills").size()).isEqualTo(totalCount - matchedCount); + int expectedScore = totalCount == 0 ? 0 : (int) Math.round(matchedCount * 100.0 / totalCount); + assertThat(matchData.path("matchScore").asInt()).isEqualTo(expectedScore); + assertSkillItemContract(matchData.path("matchedSkills").get(0)); + assertSkillItemContract(matchData.path("missingSkills").get(0)); + + JsonNode learningPath = successExample(paths.path(LEARNING_PATH).path("get")); + JsonNode learningData = learningPath.path("data"); + assertFields(learningData, Set.of("total", "items")); + assertThat(learningData.path("total").asInt()).isEqualTo(learningData.path("items").size()); + assertFields(learningData.path("items").get(0), + Set.of("priority", "skillId", "skillName", "categoryId", "categoryName", "suggestion")); + } + + private JsonNode successExample(JsonNode get) throws Exception { + JsonNode value = get.path("responses").path("200").path("content") + .path("application/json").path("examples").path("success").path("value"); + assertThat(value.isMissingNode()).isFalse(); + return value.isTextual() ? objectMapper.readTree(value.asText()) : value; + } + + private void assertSkillItemContract(JsonNode item) { + assertThat(item.isMissingNode()).isFalse(); + assertFields(item, Set.of( + "rank", "skillId", "skillCode", "nameZh", "nameEn", "sourceType", "sourceCode", + "categoryId", "categoryName", "heat", "heatSourceCode", "heatElementId", "heatProxy")); + } + + private void assertFields(JsonNode object, Set expectedFields) { + Set actualFields = new HashSet<>(); + object.fieldNames().forEachRemaining(actualFields::add); + assertThat(actualFields).containsExactlyInAnyOrderElementsOf(expectedFields); + } + private JsonNode parameter(JsonNode parameters, String name) { + for (JsonNode parameter : parameters) { + if (name.equals(parameter.path("name").asText())) { + return parameter; + } + } + return objectMapper.missingNode(); + } + + + private String studentToken() { + return token(studentId, "STUDENT"); + } + + private String token(long userId, String role) { + return "Bearer " + jwtTokenProvider.generateAccessToken( + userId, + "1380013" + String.format("%04d", userId), + Map.of("roles", role) + ); + } + + private SkillCloudResponse cloudResponse() { + SkillCloudItem skill = skill(); + return new SkillCloudResponse( + new SkillCloudNode(1L, "INTERNET", "互联网"), + new SkillCloudNode(1L, "FRONTEND_DEVELOPER", "前端开发工程师"), + null, + 1, + List.of(skill) + ); + } + + private SkillMatchResponse matchResponse() { + return new SkillMatchResponse(100, 1, 1, List.of(skill()), List.of()); + } + + private SkillLearningPathResponse learningPathResponse() { + return new SkillLearningPathResponse( + 1, + List.of(new SkillLearningPathItem( + 1, + 3L, + "JavaScript", + 101L, + "编程语言", + "先掌握 JavaScript 的基础语法和常见数据结构，再完成至少一个编码练习。" + )) + ); + } + + private SkillCloudItem skill() { + return new SkillCloudItem( + 1, + 3L, + "JAVASCRIPT", + "JavaScript", + "JavaScript", + "ESCO", + "http://data.europa.eu/esco/skill/javascript", + 101L, + "编程语言", + new BigDecimal("4.38"), + "15-1254.00", + "2.B.3.e", + false + ); + } +} diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudEndToEndIntegrationTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudEndToEndIntegrationTest.java new file mode 100644 index 0000000..3399e5b --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudEndToEndIntegrationTest.java @@ -0,0 +1,451 @@ +package com.mydreamjob.skill; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.mydreamjob.auth.JwtTokenProvider; +import com.mydreamjob.auth.User; +import com.mydreamjob.auth.UserRepository; +import com.mydreamjob.profile.StudentProfile; +import com.mydreamjob.profile.StudentProfileRepository; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.data.redis.core.ValueOperations; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; +import org.springframework.test.web.servlet.MockMvc; + +import java.math.BigDecimal; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.atomic.AtomicLong; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; +import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; + +/** + * T9: real HTTP -> Security -> Controller -> Service -> Repository -> Flyway/H2 regression. + * + *

The test deliberately uses the real V36 data and real persisted student profile. It does + * not mock the skill services, so it can catch inconsistent TOP10 derivation, accidental writes, + * and response-layer privacy leaks.

+ */ +@SpringBootTest +@AutoConfigureMockMvc +@ActiveProfiles("integration-test") +@TestPropertySource(properties = { + "spring.datasource.url=jdbc:h2:mem:skill_cloud_t9_e2e;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE", + "spring.jpa.show-sql=false" +}) +@org.springframework.transaction.annotation.Transactional +class SkillCloudEndToEndIntegrationTest { + + private static final String WORD_CLOUD_PATH = "/skills/word-cloud"; + private static final String MATCH_PATH = "/skills/match"; + private static final String LEARNING_PATH = "/skills/learning-path"; + + private static final long BACKEND_INDUSTRY_ID = 1L; + private static final long BACKEND_OCCUPATION_ID = 2L; + private static final long THREE_SKILL_INDUSTRY_ID = 3L; + private static final long THREE_SKILL_OCCUPATION_ID = 9L; + private static final long THREE_SKILL_CATEGORY_ID = 903L; + private static final long EMPTY_CATEGORY_ID = 999_001L; + + private static final AtomicLong PHONE = new AtomicLong(940_000_000L); + + @Autowired + private MockMvc mockMvc; + + @Autowired + private ObjectMapper objectMapper; + + @Autowired + private JdbcTemplate jdbcTemplate; + + @Autowired + private JwtTokenProvider jwtTokenProvider; + + @Autowired + private UserRepository userRepository; + + @Autowired + private StudentProfileRepository studentProfileRepository; + + @MockBean + private RedisTemplate redisTemplate; + + @BeforeEach + @SuppressWarnings("unchecked") + void setUpRedisMock() { + ValueOperations operations = mock(ValueOperations.class); + when(redisTemplate.opsForValue()).thenReturn(operations); + when(redisTemplate.hasKey(any())).thenReturn(false); + } + + @Test + void threeEndpointsShareTop10AndDoNotWriteForBackendDeveloper() throws Exception { + StudentFixture student = createStudent("[\"Django\",\"Git\",\"Java\"]"); + Map countsBefore = taxonomyAndProfileCounts(); + + JsonNode cloud = data(getOkBody( + WORD_CLOUD_PATH, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null)); + List allSkills = nodes(cloud.path("items")); + assertThat(cloud.path("total").asInt()).isEqualTo(16); + assertThat(allSkills).hasSize(16); + assertRanksAreContinuous(allSkills); + assertHeatOrder(allSkills); + + List expectedTop10 = allSkills.subList(0, Math.min(10, allSkills.size())); + + JsonNode match = data(getOkBody( + MATCH_PATH, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null)); + assertMatchReusesTop10(match, expectedTop10); + + JsonNode learningPath = data(getOkBody( + LEARNING_PATH, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null)); + assertLearningPathFollowsMissingSkills(learningPath, match.path("missingSkills")); + + JsonNode cloudAgain = data(getOkBody( + WORD_CLOUD_PATH, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null)); + JsonNode matchAgain = data(getOkBody( + MATCH_PATH, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null)); + JsonNode learningAgain = data(getOkBody( + LEARNING_PATH, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null)); + + assertThat(cloudAgain).isEqualTo(cloud); + assertThat(matchAgain).isEqualTo(match); + assertThat(learningAgain).isEqualTo(learningPath); + assertThat(taxonomyAndProfileCounts()).isEqualTo(countsBefore); + } + + @Test + void matchScoreUsesActualTopSizeAndRoundsToOneThird() throws Exception { + StudentFixture student = createStudent("[\"Articulate Storyline\"]"); + + JsonNode cloud = data(getOkBody( + WORD_CLOUD_PATH, + THREE_SKILL_INDUSTRY_ID, + THREE_SKILL_OCCUPATION_ID, + THREE_SKILL_CATEGORY_ID, + student.authorization(), + null)); + List skills = nodes(cloud.path("items")); + assertThat(cloud.path("total").asInt()).isEqualTo(3); + assertThat(skills).hasSize(3); + assertHeatOrder(skills); + JsonNode lastHeat = skills.get(skills.size() - 1).path("heat"); + assertThat(lastHeat.isMissingNode() || lastHeat.isNull()).isTrue(); + + JsonNode match = data(getOkBody( + MATCH_PATH, + THREE_SKILL_INDUSTRY_ID, + THREE_SKILL_OCCUPATION_ID, + THREE_SKILL_CATEGORY_ID, + student.authorization(), + null)); + assertThat(match.path("totalCount").asInt()).isEqualTo(3); + assertThat(match.path("matchedCount").asInt()).isEqualTo(1); + assertThat(match.path("matchScore").asInt()).isEqualTo(33); + assertThat(match.path("matchedSkills")).hasSize(1); + assertThat(match.path("missingSkills")).hasSize(2); + assertThat(ids(match.path("matchedSkills"))) + .containsExactly(skills.get(0).path("skillId").asLong()); + + JsonNode learningPath = data(getOkBody( + LEARNING_PATH, + THREE_SKILL_INDUSTRY_ID, + THREE_SKILL_OCCUPATION_ID, + THREE_SKILL_CATEGORY_ID, + student.authorization(), + null)); + assertLearningPathFollowsMissingSkills(learningPath, match.path("missingSkills")); + } + + @Test + void emptyCategoryReturnsEmptyResponsesAtPersistenceBoundary() throws Exception { + StudentFixture student = createStudent("[\"Java\"]"); + jdbcTemplate.update( + "INSERT INTO skill_categories " + + "(id, occupation_id, code, name_zh, suggestion_template_code, sort_order, created_at, updated_at) " + + "VALUES (?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)", + EMPTY_CATEGORY_ID, + BACKEND_OCCUPATION_ID, + "T9_EMPTY_CATEGORY", + "T9 空分类", + "GENERAL", + 99); + + JsonNode cloud = data(getOkBody( + WORD_CLOUD_PATH, + BACKEND_INDUSTRY_ID, + BACKEND_OCCUPATION_ID, + EMPTY_CATEGORY_ID, + student.authorization(), + null)); + assertThat(cloud.path("total").asInt()).isZero(); + assertThat(cloud.path("items")).isEmpty(); + + JsonNode match = data(getOkBody( + MATCH_PATH, + BACKEND_INDUSTRY_ID, + BACKEND_OCCUPATION_ID, + EMPTY_CATEGORY_ID, + student.authorization(), + null)); + assertThat(match.path("matchScore").asInt()).isZero(); + assertThat(match.path("matchedCount").asInt()).isZero(); + assertThat(match.path("totalCount").asInt()).isZero(); + assertThat(match.path("matchedSkills")).isEmpty(); + assertThat(match.path("missingSkills")).isEmpty(); + + JsonNode learningPath = data(getOkBody( + LEARNING_PATH, + BACKEND_INDUSTRY_ID, + BACKEND_OCCUPATION_ID, + EMPTY_CATEGORY_ID, + student.authorization(), + null)); + assertThat(learningPath.path("total").asInt()).isZero(); + assertThat(learningPath.path("items")).isEmpty(); + } + + @Test + void identityQueryParametersCannotOverrideAuthenticatedStudent() throws Exception { + StudentFixture student = createStudent("[\"Java\"]"); + User company = createUser(User.Role.COMPANY); + + for (String path : List.of(WORD_CLOUD_PATH, MATCH_PATH, LEARNING_PATH)) { + JsonNode authenticated = data(getOkBody( + path, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null)); + JsonNode withOverrides = data(getOkBody( + path, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), company.getId())); + + assertThat(withOverrides).isEqualTo(authenticated); + } + } + + @Test + void responsesDoNotLeakPrivateProfileDataOrInternalErrors() throws Exception { + StudentFixture student = createStudent("[\"Java\"]"); + + for (String path : List.of(WORD_CLOUD_PATH, MATCH_PATH, LEARNING_PATH)) { + String body = getOkBody( + path, BACKEND_INDUSTRY_ID, BACKEND_OCCUPATION_ID, null, student.authorization(), null); + assertNoPrivateData(body, student); + } + + String errorBody = mockMvc.perform(get(WORD_CLOUD_PATH) + .param("industryId", String.valueOf(BACKEND_INDUSTRY_ID)) + .param("occupationId", String.valueOf(BACKEND_OCCUPATION_ID)) + .param("categoryId", "999999") + .header("Authorization", student.authorization())) + .andExpect(status().isBadRequest()) + .andExpect(jsonPath("$.code").isNumber()) + .andReturn().getResponse().getContentAsString(StandardCharsets.UTF_8); + + assertThat(errorBody.toLowerCase(Locale.ROOT)) + .doesNotContain("exception") + .doesNotContain("org.springframework") + .doesNotContain("java.sql") + .doesNotContain("stacktrace") + .doesNotContain("\tat "); + } + + private void assertMatchReusesTop10(JsonNode match, List expectedTop10) { + List expectedIds = ids(expectedTop10); + List matchedIds = ids(match.path("matchedSkills")); + List missingIds = ids(match.path("missingSkills")); + + assertThat(match.path("totalCount").asInt()).isEqualTo(expectedTop10.size()); + assertThat(match.path("matchedCount").asInt()).isEqualTo(matchedIds.size()); + assertThat(matchedIds).allMatch(expectedIds::contains); + assertThat(expectedIds).containsAll(matchedIds); + + List expectedMatched = expectedIds.stream().filter(matchedIds::contains).toList(); + List expectedMissing = expectedIds.stream().filter(id -> !matchedIds.contains(id)).toList(); + assertThat(matchedIds).containsExactlyElementsOf(expectedMatched); + assertThat(missingIds).containsExactlyElementsOf(expectedMissing); + + List union = new ArrayList<>(matchedIds); + union.addAll(missingIds); + assertThat(union).containsExactlyInAnyOrderElementsOf(expectedIds); + int expectedScore = expectedTop10.isEmpty() + ? 0 + : (int) Math.round(matchedIds.size() * 100.0 / expectedTop10.size()); + assertThat(match.path("matchScore").asInt()).isEqualTo(expectedScore); + } + + private void assertLearningPathFollowsMissingSkills(JsonNode learningPath, JsonNode missingSkills) { + assertThat(learningPath.path("total").asInt()).isEqualTo(missingSkills.size()); + assertThat(learningPath.path("items")).hasSize(missingSkills.size()); + for (int index = 0; index < missingSkills.size(); index++) { + JsonNode item = learningPath.path("items").get(index); + JsonNode missing = missingSkills.get(index); + assertThat(item.path("priority").asInt()).isEqualTo(index + 1); + assertThat(item.path("skillId").asLong()).isEqualTo(missing.path("skillId").asLong()); + assertThat(item.path("skillName").asText()).isEqualTo(missing.path("nameZh").asText()); + assertThat(item.path("suggestion").asText()).isNotBlank(); + } + } + + private void assertRanksAreContinuous(List items) { + for (int index = 0; index < items.size(); index++) { + assertThat(items.get(index).path("rank").asInt()).isEqualTo(index + 1); + } + } + + private void assertHeatOrder(List items) { + boolean seenNullHeat = false; + BigDecimal previousHeat = null; + for (JsonNode item : items) { + if (item.path("heat").isNull()) { + seenNullHeat = true; + continue; + } + assertThat(seenNullHeat).isFalse(); + BigDecimal currentHeat = item.path("heat").decimalValue(); + if (previousHeat != null) { + assertThat(currentHeat).isLessThanOrEqualTo(previousHeat); + } + previousHeat = currentHeat; + } + } + + private void assertNoPrivateData(String body, StudentFixture student) { + assertThat(body).doesNotContain("skills_json"); + assertThat(body).doesNotContain(student.phone()); + assertThat(body).doesNotContain(student.email()); + assertThat(body).doesNotContain(student.resumeUrl()); + assertThat(body).doesNotContain(student.resumeOriginalName()); + assertThat(body).doesNotContain(student.accessToken()); + } + + private String getOkBody(String path, + long industryId, + long occupationId, + Long categoryId, + String authorization, + Long overrideIdentity) throws Exception { + var request = get(path) + .param("industryId", String.valueOf(industryId)) + .param("occupationId", String.valueOf(occupationId)) + .header("Authorization", authorization); + if (categoryId != null) { + request.param("categoryId", String.valueOf(categoryId)); + } + if (overrideIdentity != null) { + request.param("userId", String.valueOf(overrideIdentity)); + request.param("studentId", String.valueOf(overrideIdentity)); + } + + return mockMvc.perform(request) + .andExpect(status().isOk()) + .andExpect(jsonPath("$.code").value(200)) + .andReturn().getResponse().getContentAsString(StandardCharsets.UTF_8); + } + + private JsonNode data(String body) throws Exception { + return objectMapper.readTree(body).path("data"); + } + + private List nodes(JsonNode array) { + List nodes = new ArrayList<>(); + array.forEach(nodes::add); + return nodes; + } + + private List ids(JsonNode array) { + return ids(nodes(array)); + } + + private List ids(List items) { + return items.stream().map(node -> node.path("skillId").asLong()).toList(); + } + + private StudentFixture createStudent(String skillsJson) { + long phoneNumber = PHONE.getAndIncrement(); + String phone = String.valueOf(phoneNumber); + String email = "private.student." + phoneNumber + "@example.com"; + String resumeUrl = "https://private.example/" + phoneNumber + "-resume.pdf"; + String resumeOriginalName = "private-" + phoneNumber + "-resume.pdf"; + + User user = userRepository.saveAndFlush(User.builder() + .phone(phone) + .email(email) + .role(User.Role.STUDENT) + .roles(Set.of(User.Role.STUDENT)) + .active(true) + .verified(true) + .build()); + studentProfileRepository.saveAndFlush(StudentProfile.builder() + .userId(user.getId()) + .realName("sensitive-real-name") + .bio("raw-resume-sentinel") + .resumeUrl(resumeUrl) + .resumeOriginalName(resumeOriginalName) + .skillsJson(skillsJson) + .build()); + + String accessToken = jwtTokenProvider.generateAccessToken( + user.getId(), + phone, + Map.of("roles", "STUDENT")); + return new StudentFixture(user.getId(), phone, email, resumeUrl, resumeOriginalName, accessToken); + } + + private User createUser(User.Role role) { + long phoneNumber = PHONE.getAndIncrement(); + return userRepository.saveAndFlush(User.builder() + .phone(String.valueOf(phoneNumber)) + .role(role) + .roles(Set.of(role)) + .active(true) + .verified(true) + .build()); + } + + private Map taxonomyAndProfileCounts() { + Map counts = new LinkedHashMap<>(); + for (String table : List.of( + "skill_source_versions", + "skill_industries", + "skill_occupations", + "skill_categories", + "skills", + "skill_aliases", + "occupation_skills", + "student_profiles")) { + Long count = jdbcTemplate.queryForObject("SELECT COUNT(*) FROM " + table, Long.class); + counts.put(table, count); + } + return counts; + } + + private record StudentFixture( + Long userId, + String phone, + String email, + String resumeUrl, + String resumeOriginalName, + String accessToken + ) { + String authorization() { + return "Bearer " + accessToken; + } + } +} diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudQueryServiceTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudQueryServiceTest.java new file mode 100644 index 0000000..c207304 --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudQueryServiceTest.java @@ -0,0 +1,325 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.common.exception.BusinessException; +import com.mydreamjob.common.exception.ErrorCode; +import com.mydreamjob.skill.domain.SkillIndustry; +import com.mydreamjob.skill.domain.SkillOccupation; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillCloudResponse; +import com.mydreamjob.skill.dto.SkillCloudSkillProjection; +import org.assertj.core.api.ThrowableAssert.ThrowingCallable; +import org.hibernate.boot.model.relational.Namespace; +import org.hibernate.boot.model.relational.Sequence; +import org.hibernate.mapping.Table; +import org.hibernate.tool.schema.spi.SchemaFilter; +import org.hibernate.tool.schema.spi.SchemaFilterProvider; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; +import com.mydreamjob.skill.repository.OccupationSkillRepository; +import com.mydreamjob.skill.repository.SkillCategoryRepository; +import com.mydreamjob.skill.repository.SkillIndustryRepository; +import com.mydreamjob.skill.repository.SkillOccupationRepository; +import org.springframework.transaction.annotation.Transactional; + +import java.math.BigDecimal; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import java.util.stream.IntStream; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowableOfType; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +@SpringBootTest +@TestPropertySource(properties = { + "spring.datasource.url=jdbc:h2:mem:integrationdb_skill_query;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE", + "spring.jpa.hibernate.ddl-auto=validate", + "spring.jpa.properties.hibernate.hbm2ddl.schema_filter_provider=com.mydreamjob.skill.SkillCloudQueryServiceTest$SkillQuerySchemaFilterProvider" +}) +@ActiveProfiles("integration-test") +@Transactional +class SkillCloudQueryServiceTest { + + @Autowired + private SkillCloudQueryService service; + + @Autowired + private JdbcTemplate jdbcTemplate; + + @MockBean + private RedisTemplate redisTemplate; + + @Test + void returnsAllSeedSkillsForIndustryAndOccupationWithContinuousRanks() { + SkillCloudResponse response = service.query(1L, 1L, null); + + assertThat(response.industry().id()).isEqualTo(1L); + assertThat(response.industry().code()).isEqualTo("INTERNET"); + assertThat(response.industry().name()).isEqualTo("互联网"); + assertThat(response.occupation().id()).isEqualTo(1L); + assertThat(response.occupation().code()).isEqualTo("FRONTEND_DEVELOPER"); + assertThat(response.occupation().name()).isEqualTo("前端开发工程师"); + assertThat(response.category()).isNull(); + assertThat(response.total()).isEqualTo(15); + assertThat(response.items()).hasSize(15); + assertThat(response.items()) + .extracting(SkillCloudItem::rank) + .containsExactlyElementsOf(IntStream.rangeClosed(1, 15).boxed().toList()); + assertThat(response.items()) + .extracting(SkillCloudItem::skillId) + .doesNotHaveDuplicates(); + } + + @Test + void sortsByHeatThenNameAndKeepsNullHeatLast() { + SkillCloudResponse response = service.query(1L, 1L, 104L); + + assertThat(response.category().id()).isEqualTo(104L); + assertThat(response.category().code()).isEqualTo("TOOLS"); + assertThat(response.category().name()).isEqualTo("工程化工具"); + assertThat(response.items()) + .extracting(SkillCloudItem::nameZh) + .containsExactly("Gradle", "Node.js", "Figma"); + assertThat(response.items()) + .extracting(SkillCloudItem::heat) + .containsExactly(new BigDecimal("4.88"), new BigDecimal("4.88"), null); + assertThat(response.items().get(response.items().size() - 1).heat()).isNull(); + + SkillCloudResponse all = service.query(1L, 1L, null); + assertThat(all.items().stream() + .map(SkillCloudItem::heat) + .filter(java.util.Objects::nonNull) + .toList()) + .isSortedAccordingTo(java.util.Comparator.reverseOrder()); + assertThat(all.items().get(all.items().size() - 1).heat()).isNull(); + assertThat(all.items().subList(0, all.items().size() - 1)) + .allSatisfy(item -> assertThat(item.heat()).isNotNull()); + } + + @Test + void comparatorUsesSkillIdWhenEqualKeysArriveInReverseOrder() { + SkillIndustry industry = new SkillIndustry(); + industry.setId(1L); + industry.setCode("INTERNET"); + industry.setNameZh("互联网"); + + SkillOccupation occupation = new SkillOccupation(); + occupation.setId(1L); + occupation.setIndustryId(1L); + occupation.setCode("FRONTEND_DEVELOPER"); + occupation.setNameZh("前端开发工程师"); + + SkillIndustryRepository industryRepository = mock(SkillIndustryRepository.class); + when(industryRepository.findById(1L)).thenReturn(Optional.of(industry)); + + SkillOccupationRepository occupationRepository = mock(SkillOccupationRepository.class); + when(occupationRepository.findById(1L)).thenReturn(Optional.of(occupation)); + + OccupationSkillRepository occupationSkillRepository = mock(OccupationSkillRepository.class); + when(occupationSkillRepository.findSkillProjectionsByOccupationId(1L)) + .thenReturn(List.of( + projection(9002L, "TIE_SKILL_HIGH"), + projection(9001L, "TIE_SKILL_LOW"))); + + SkillCloudQueryService isolatedService = new SkillCloudQueryService( + industryRepository, + occupationRepository, + mock(SkillCategoryRepository.class), + occupationSkillRepository); + + SkillCloudResponse response = isolatedService.query(1L, 1L, null); + + assertThat(response.items()) + .extracting(SkillCloudItem::skillId) + .containsExactly(9001L, 9002L); + } + + private SkillCloudSkillProjection projection(Long skillId, String skillCode) { + return new SkillCloudSkillProjection( + skillId, + skillCode, + "同名技能", + "Tie Skill", + "O*NET", + "TEST|Tie Skill", + 104L, + "工程化工具", + new BigDecimal("4.88"), + "15-1254.00", + "4.A.3.b.1", + true, + 1, + 1 + ); + } + + @Test + void categoryFilterReturnsOnlyTheRequestedCategoryAndOmittedCategoryReturnsMore() { + SkillCloudResponse all = service.query(1L, 1L, null); + SkillCloudResponse filtered = service.query(1L, 1L, 101L); + + assertThat(filtered.total()).isEqualTo(2); + assertThat(filtered.items()) + .extracting(SkillCloudItem::categoryId) + .containsOnly(101L); + assertThat(filtered.items()) + .extracting(SkillCloudItem::skillId) + .containsExactlyInAnyOrder(3L, 4L); + assertThat(all.total()).isGreaterThan(filtered.total()); + } + + @Test + void returnsEmptySuccessForLegalCategoryWithoutSkills() { + jdbcTemplate.update(""" + INSERT INTO skill_categories + (id, occupation_id, code, name_zh, suggestion_template_code, sort_order) + VALUES + (9101, 1, 'EMPTY_CATEGORY', '空分类', 'GENERAL', 99) + """); + + SkillCloudResponse response = service.query(1L, 1L, 9101L); + + assertThat(response.category().id()).isEqualTo(9101L); + assertThat(response.category().name()).isEqualTo("空分类"); + assertThat(response.total()).isZero(); + assertThat(response.items()).isEmpty(); + } + + @Test + void mapsSkillAndHeatProvenanceFieldsFromTheDatabase() { + SkillCloudResponse frameworks = service.query(1L, 1L, 103L); + SkillCloudItem react = frameworks.items().stream() + .filter(item -> item.skillId().equals(26L)) + .findFirst() + .orElseThrow(); + + Map row = jdbcTemplate.queryForMap(""" + SELECT os.importance, os.importance_source_code, os.importance_element_id, + os.importance_is_proxy + FROM occupation_skills os + WHERE os.occupation_id = 1 AND os.skill_id = 26 + """); + + assertThat(react.rank()).isEqualTo(1); + assertThat(react.skillCode()).isEqualTo("REACT"); + assertThat(react.nameZh()).isEqualTo("React"); + assertThat(react.nameEn()).isEqualTo("React"); + assertThat(react.categoryId()).isEqualTo(103L); + assertThat(react.categoryName()).isEqualTo("前端框架"); + assertThat(react.sourceType()).isEqualTo("O*NET"); + assertThat(react.sourceCode()).isEqualTo("15-1254.00|React|2.E.7.h"); + assertThat(react.heat()).isEqualByComparingTo((BigDecimal) row.get("importance")); + assertThat(react.heatSourceCode()).isEqualTo(row.get("importance_source_code")); + assertThat(react.heatElementId()).isEqualTo(row.get("importance_element_id")); + assertThat(react.heatProxy()).isEqualTo(toBoolean(row.get("importance_is_proxy"))); + + SkillCloudResponse languages = service.query(1L, 1L, 101L); + SkillCloudItem javascript = languages.items().stream() + .filter(item -> item.skillId().equals(3L)) + .findFirst() + .orElseThrow(); + + assertThat(javascript.sourceType()).isEqualTo("ESCO"); + assertThat(javascript.sourceCode()) + .isEqualTo("http://data.europa.eu/esco/skill/3cd569a2-4f88-4c1e-9995-8dce8c5e51a7"); + assertThat(javascript.heatSourceCode()).isEqualTo("15-1254.00"); + } + + @Test + void rejectsMissingOrNonPositiveIdentifiersWithBadRequest() { + assertBadRequest(() -> service.query(null, 1L, null), "industryId"); + assertBadRequest(() -> service.query(0L, 1L, null), "industryId"); + assertBadRequest(() -> service.query(-1L, 1L, null), "industryId"); + assertBadRequest(() -> service.query(1L, null, null), "occupationId"); + assertBadRequest(() -> service.query(1L, 0L, null), "occupationId"); + assertBadRequest(() -> service.query(1L, -1L, null), "occupationId"); + assertBadRequest(() -> service.query(1L, 1L, 0L), "categoryId"); + assertBadRequest(() -> service.query(1L, 1L, -1L), "categoryId"); + } + + @Test + void rejectsUnknownOrMismatchedTaxonomyNodesWithBadRequest() { + assertBadRequest(() -> service.query(999L, 1L, null), "industry not found"); + assertBadRequest(() -> service.query(1L, 999L, null), "occupation not found"); + assertBadRequest(() -> service.query(2L, 1L, null), "occupation does not belong"); + assertBadRequest(() -> service.query(1L, 1L, 201L), "category does not belong"); + assertBadRequest(() -> service.query(1L, 1L, 9999L), "category not found"); + } + + private boolean toBoolean(Object value) { + if (value instanceof Boolean booleanValue) { + return booleanValue; + } + return ((Number) value).intValue() != 0; + } + + private void assertBadRequest(ThrowingCallable action, String detailFragment) { + BusinessException exception = catchThrowableOfType(action, BusinessException.class); + assertThat(exception).isNotNull(); + assertThat(exception.getErrorCodeInt()).isEqualTo(ErrorCode.BAD_REQUEST.getCode()); + assertThat(exception.getErrorMessage()).contains(detailFragment); + } + + public static class SkillQuerySchemaFilterProvider implements SchemaFilterProvider { + + private static final Set VALIDATED_TABLES = Set.of( + "skill_source_versions", + "skill_industries", + "skill_occupations", + "skill_categories", + "skills", + "skill_aliases", + "occupation_skills" + ); + + @Override + public SchemaFilter getCreateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getDropFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getTruncatorFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getMigrateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getValidateFilter() { + return new SchemaFilter() { + @Override + public boolean includeNamespace(Namespace namespace) { + return true; + } + + @Override + public boolean includeTable(Table table) { + return VALIDATED_TABLES.contains(table.getName().toLowerCase(Locale.ROOT)); + } + + @Override + public boolean includeSequence(Sequence sequence) { + return false; + } + }; + } + } +} diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudSeedDataIntegrationTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudSeedDataIntegrationTest.java new file mode 100644 index 0000000..4df5e59 --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillCloudSeedDataIntegrationTest.java @@ -0,0 +1,1036 @@ +package com.mydreamjob.skill; + +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.core.io.ClassPathResource; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; + +import java.io.BufferedReader; +import java.io.InputStream; +import java.io.InputStreamReader; +import java.math.BigDecimal; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TreeSet; +import java.util.regex.Pattern; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.assertThrows; + +@SpringBootTest +@TestPropertySource(properties = + "spring.datasource.url=jdbc:h2:mem:integrationdb_skill_seed;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE") +@ActiveProfiles("integration-test") +class SkillCloudSeedDataIntegrationTest { + + private static final String EVIDENCE_RESOURCE = "skill-cloud/onet-31-reference.csv"; + private static final String EVIDENCE_HEADER = + "evidence_type,source_soc,element_id,data_value,workplace_example,source_file"; + private static final BigDecimal IMPORTANCE_TOLERANCE = new BigDecimal("0.001"); + private static final Set IMPORTANCE_SOURCE_FILES = Set.of( + "work_activities.csv", "knowledge.csv", "essential_skills.csv", "transferable_skills.csv"); + + private static final Pattern ONET_SKILL_SOURCE_PATTERN = + Pattern.compile("^\\d{2}-\\d{4}\\.\\d{2}\\|[^|]+\\|2\\.E\\.[0-9.a-z]+$"); + + private static final Map> EXPECTED_CATEGORIES = Map.ofEntries( + Map.entry("FRONTEND_DEVELOPER", List.of( + "编程语言", "Web 基础", "前端框架", "工程化工具", "浏览器与调试", "测试", "版本控制与协作")), + Map.entry("BACKEND_DEVELOPER", List.of( + "编程语言", "后端框架与平台", "数据库", "缓存与消息", "云/DevOps", "测试", "工程化工具", "安全")), + Map.entry("QA_ENGINEER", List.of( + "编程语言", "测试设计", "自动化测试框架", "接口与数据库测试", "性能测试", "CI/CD 与工程化工具")), + Map.entry("DEVOPS_ENGINEER", List.of( + "编程语言与脚本", "云平台", "容器与编排", "CI/CD", "配置管理与 IaC", "可观测性", "安全", "数据库")), + Map.entry("FINANCIAL_DATA_ANALYST", List.of( + "编程语言", "数据分析与可视化", "数据库与数据仓库", "统计与建模", "金融计算框架/平台", "金融领域知识", "工程化工具")), + Map.entry("RISK_MODELING_ENGINEER", List.of( + "编程语言", "统计与建模", "风险模型", "建模框架/平台", "数据处理与数据库", "模型验证", "部署与工程化工具")), + Map.entry("QUANTITATIVE_DEVELOPER", List.of( + "编程语言", "量化建模框架/平台", "市场与金融数据", "数据库", "低延迟与高性能工程", "测试", "云/DevOps")), + Map.entry("EDUCATION_PRODUCT_MANAGER", List.of( + "教育领域知识", "产品分析方法", "需求与流程", "数据分析", "教育平台与工具", "协作与工程化工具")), + Map.entry("INSTRUCTIONAL_DESIGNER", List.of( + "教学设计", "课程与学习平台", "数字化内容工具", "评估设计", "学习数据分析", "无障碍与规范")), + Map.entry("EDUCATION_DATA_ANALYST", List.of( + "编程语言", "数据分析与可视化", "数据库与 SQL", "统计与建模", "教育领域知识", "报表与工程化工具")) + ); + + private static final Map EXPECTED_ESCO_OCCUPATION_SOURCES = Map.ofEntries( + Map.entry("FRONTEND_DEVELOPER", + "http://data.europa.eu/esco/occupation/c40a2919-48a9-40ea-b506-1f34f693496d"), + Map.entry("BACKEND_DEVELOPER", + "http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1"), + Map.entry("QA_ENGINEER", + "http://data.europa.eu/esco/occupation/106f79e4-6264-45f1-9e7a-297435cd684b"), + Map.entry("DEVOPS_ENGINEER", + "http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1"), + Map.entry("FINANCIAL_DATA_ANALYST", + "http://data.europa.eu/esco/occupation/8d586ee9-0ab1-4155-a3b0-2ca786c8e48c"), + Map.entry("RISK_MODELING_ENGINEER", + "http://data.europa.eu/esco/occupation/b09acdda-6680-49d1-a411-129610ee3bc0"), + Map.entry("QUANTITATIVE_DEVELOPER", + "http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1"), + Map.entry("EDUCATION_PRODUCT_MANAGER", + "http://data.europa.eu/esco/occupation/a79170ad-0b7b-4697-864f-1748359aa10a"), + Map.entry("INSTRUCTIONAL_DESIGNER", + "http://data.europa.eu/esco/occupation/a9c30651-ccbc-4a80-900c-7880615cdf6e"), + Map.entry("EDUCATION_DATA_ANALYST", + "http://data.europa.eu/esco/occupation/d3edb8f8-3a06-47a0-8fb9-9b212c006aa2") + ); + + @Autowired + private JdbcTemplate jdbcTemplate; + + @MockBean + private RedisTemplate redisTemplate; + + @Test + void seedDataCoversIndustriesOccupationsAndEveryDeclaredCategory() { + assertEquals(3L, count("SELECT COUNT(*) FROM skill_industries")); + assertEquals(10L, count("SELECT COUNT(*) FROM skill_occupations")); + assertEquals(68L, count("SELECT COUNT(*) FROM skill_categories")); + + for (Map.Entry> entry : EXPECTED_CATEGORIES.entrySet()) { + String occupationCode = entry.getKey(); + List actualCategories = jdbcTemplate.queryForList( + "SELECT c.name_zh " + + "FROM skill_categories c " + + "JOIN skill_occupations o ON o.id = c.occupation_id " + + "WHERE o.code = ? " + + "ORDER BY c.sort_order, c.id", + String.class, occupationCode); + assertEquals(entry.getValue(), actualCategories, + "Category coverage mismatch for " + occupationCode); + + for (String categoryName : entry.getValue()) { + long relations = count( + "SELECT COUNT(*) FROM occupation_skills os " + + "JOIN skill_occupations o ON o.id = os.occupation_id " + + "JOIN skill_categories c ON c.id = os.category_id " + + "WHERE o.code = ? AND c.name_zh = ?", + occupationCode, categoryName); + assertTrue(relations >= 1L, + "Category has no skill relation: occupation=" + occupationCode + + ", category=" + categoryName + ", actualRelations=" + relations); + } + + Map coverage = jdbcTemplate.queryForMap( + "SELECT COUNT(os.skill_id) AS skill_count, " + + "SUM(CASE WHEN os.importance IS NOT NULL THEN 1 ELSE 0 END) AS weighted_count " + + "FROM skill_occupations o " + + "LEFT JOIN occupation_skills os ON os.occupation_id = o.id " + + "WHERE o.code = ? " + + "GROUP BY o.id", + occupationCode); + assertTrue(number(coverage.get("skill_count")) >= 10, + occupationCode + " must have at least ten skills"); + assertTrue(number(coverage.get("weighted_count")) >= 10, + occupationCode + " must have at least ten weighted skills"); + } + } + + @Test + void occupationNodesUseApprovedEscoUrisAndOnetSkillSourcesAreTraceable() { + assertEquals(10L, count( + "SELECT COUNT(*) FROM skill_occupations " + + "WHERE source_version_id = 1 " + + "AND source_code LIKE 'http://data.europa.eu/esco/occupation/%'")); + + for (Map.Entry entry : EXPECTED_ESCO_OCCUPATION_SOURCES.entrySet()) { + assertEquals(1L, count( + "SELECT COUNT(*) FROM skill_occupations o " + + "JOIN skill_source_versions sv ON sv.id = o.source_version_id " + + "WHERE o.code = ? AND o.source_code = ? AND sv.source_name = 'ESCO'", + entry.getKey(), entry.getValue()), + "Unexpected ESCO source for " + entry.getKey()); + } + + Map snapshot = loadEvidence("SOFTWARE"); + List> rows = jdbcTemplate.queryForList( + "SELECT id, code, source_code FROM skills WHERE source_type = 'O*NET' ORDER BY id"); + assertEquals(47, rows.size(), "Unexpected O*NET skill source count"); + + for (Map row : rows) { + long skillId = number(row.get("id")); + String skillCode = (String) row.get("code"); + String sourceCode = (String) row.get("source_code"); + assertTrue(ONET_SKILL_SOURCE_PATTERN.matcher(sourceCode).matches(), + "Invalid O*NET software_skills source: skillId=" + skillId + + ", skillCode=" + skillCode + ", sourceCode=" + sourceCode); + + String[] parts = sourceCode.split("\\|", -1); + EvidenceRow evidence = snapshot.get(softwareKey(parts[0], parts[1], parts[2])); + assertNotNull(evidence, + "Missing SOFTWARE evidence snapshot: skillId=" + skillId + + ", skillCode=" + skillCode + ", expectedSourceCode=" + sourceCode); + assertEquals("software_skills.csv", evidence.sourceFile(), + "Unexpected source file for skillId=" + skillId + ", skillCode=" + skillCode); + } + + assertEquals(rows.size(), snapshot.size(), + "SOFTWARE snapshot must contain exactly the 47 referenced O*NET source triples"); + + assertOnetSkillSource("REACT", "15-1254.00|React|2.E.7.h"); + assertOnetSkillSource("GOOGLE_CHROME", "17-2051.02|Google Chrome|2.E.10.e"); + assertOnetSkillSource("APACHE_KAFKA", "15-1299.07|Apache Kafka|2.E.7.b"); + assertOnetSkillSource("GITLAB", "15-1299.08|GitLab|2.E.10.a"); + assertOnetSkillSource("ARTICULATE_STORYLINE", "25-9031.00|Articulate Storyline|2.E.8.b"); + } + + @Test + void everyWeightHasTraceableOnetProvenanceAndKnownRepresentativeValue() { + assertEquals(139L, count("SELECT COUNT(*) FROM occupation_skills")); + assertEquals(2L, count("SELECT COUNT(*) FROM occupation_skills WHERE importance IS NULL")); + assertEquals(0L, count( + "SELECT COUNT(*) FROM occupation_skills " + + "WHERE importance IS NOT NULL " + + "AND (importance_source_code IS NULL OR TRIM(importance_source_code) = '' " + + "OR importance_element_id IS NULL OR TRIM(importance_element_id) = '')")); + assertEquals(0L, count( + "SELECT COUNT(*) FROM occupation_skills " + + "WHERE importance_is_proxy IS NULL OR importance_is_proxy NOT IN (FALSE, TRUE)")); + assertEquals(0L, count( + "SELECT COUNT(*) FROM occupation_skills os " + + "JOIN skill_source_versions sv ON sv.id = os.source_version_id " + + "WHERE os.importance IS NOT NULL AND sv.source_name <> 'O*NET'")); + + Map snapshot = loadEvidence("IMPORTANCE"); + List> rows = jdbcTemplate.queryForList( + "SELECT os.id, o.code AS occupation_code, s.code AS skill_code, os.importance, " + + "os.importance_source_code, os.importance_element_id, os.importance_is_proxy " + + "FROM occupation_skills os " + + "JOIN skill_occupations o ON o.id = os.occupation_id " + + "JOIN skills s ON s.id = os.skill_id " + + "WHERE os.importance IS NOT NULL " + + "ORDER BY os.id"); + assertEquals(137, rows.size(), "Unexpected non-null Importance relation count"); + + for (Map row : rows) { + long relationId = number(row.get("id")); + String occupationCode = (String) row.get("occupation_code"); + String skillCode = (String) row.get("skill_code"); + String sourceCode = (String) row.get("importance_source_code"); + String elementId = (String) row.get("importance_element_id"); + BigDecimal importance = (BigDecimal) row.get("importance"); + int proxy = toInteger(row.get("importance_is_proxy")); + + assertTrue(proxy == 0 || proxy == 1, + "Invalid importance_is_proxy: relationId=" + relationId + + ", occupationCode=" + occupationCode + ", skillCode=" + skillCode + + ", actual=" + proxy); + assertNotNull(sourceCode, + "Missing Importance SOC: relationId=" + relationId + + ", occupationCode=" + occupationCode + ", skillCode=" + skillCode); + assertNotNull(elementId, + "Missing Importance Element ID: relationId=" + relationId + + ", occupationCode=" + occupationCode + ", skillCode=" + skillCode); + assertNotNull(importance, + "Missing Importance value: relationId=" + relationId + + ", occupationCode=" + occupationCode + ", skillCode=" + skillCode); + + EvidenceRow evidence = snapshot.get(importanceKey(sourceCode, elementId)); + assertNotNull(evidence, + "Missing IMPORTANCE evidence snapshot: relationId=" + relationId + + ", occupationCode=" + occupationCode + ", skillCode=" + skillCode + + ", expectedSource=" + sourceCode + "|" + elementId + + "|" + importance); + assertNotNull(evidence.dataValue(), + "IMPORTANCE snapshot has empty Data Value: relationId=" + relationId + + ", sourceCode=" + sourceCode + ", elementId=" + elementId); + BigDecimal difference = evidence.dataValue().subtract(importance).abs(); + assertTrue(difference.compareTo(IMPORTANCE_TOLERANCE) <= 0, + "Importance mismatch: relationId=" + relationId + + ", occupationCode=" + occupationCode + ", skillCode=" + skillCode + + ", sourceCode=" + sourceCode + ", elementId=" + elementId + + ", expected=" + evidence.dataValue() + ", actual=" + importance + + ", absoluteDifference=" + difference + + ", tolerance=" + IMPORTANCE_TOLERANCE); + } + + assertEquals(rows.stream() + .map(row -> importanceKey( + (String) row.get("importance_source_code"), + (String) row.get("importance_element_id"))) + .distinct() + .count(), + snapshot.size(), + "IMPORTANCE snapshot must contain exactly the referenced unique SOC/Element ID combinations"); + + assertImportance("FRONTEND_DEVELOPER", "JAVASCRIPT", + "15-1254.00", "4.A.3.b.1", "4.88"); + assertImportance("BACKEND_DEVELOPER", "JAVA", + "15-1252.00", "4.A.3.b.1", "4.61"); + assertImportance("QA_ENGINEER", "SELENIUM", + "15-1253.00", "4.A.3.b.1", "4.94"); + assertImportance("DEVOPS_ENGINEER", "DOCKER", + "15-1244.00", "4.A.3.b.1", "4.83"); + assertImportance("FINANCIAL_DATA_ANALYST", "PYTHON", + "13-2031.00", "4.A.3.b.1", "4.48"); + assertImportance("RISK_MODELING_ENGINEER", "PYTHON", + "13-2053.00", "4.A.3.b.1", "4.41"); + assertImportance("QUANTITATIVE_DEVELOPER", "CPP", + "13-2099.01", "4.A.3.b.1", "4.68"); + assertImportance("EDUCATION_PRODUCT_MANAGER", "PROJECT_MANAGEMENT", + "13-1082.00", "4.A.2.b.4", "3.73"); + assertImportance("INSTRUCTIONAL_DESIGNER", "INSTRUCTIONAL_DESIGN", + "25-9031.00", "2.C.6", "4.78"); + assertImportance("EDUCATION_DATA_ANALYST", "PYTHON", + "15-2051.00", "4.A.3.b.1", "4.75"); + + assertNullImportance("FRONTEND_DEVELOPER", "FIGMA"); + assertNullImportance("INSTRUCTIONAL_DESIGNER", "ADOBE_PHOTOSHOP"); + } + + @Test + void evidenceLookupRejectsMissingOrTamperedOnetRows() { + Map software = loadEvidence("SOFTWARE"); + assertTrue(hasSoftware(software, "15-1254.00", "React", "2.E.7.h"), + "Known SOFTWARE evidence row should be present"); + assertFalse(hasSoftware(software, "99-9999.00", "React", "2.E.7.h"), + "Missing SOC must not match SOFTWARE evidence"); + assertFalse(hasSoftware(software, "15-1254.00", "Not React", "2.E.7.h"), + "Missing workplace example must not match SOFTWARE evidence"); + assertFalse(hasSoftware(software, "15-1254.00", "React", "9.Z.9.z"), + "Missing Element ID must not match SOFTWARE evidence"); + + Map importance = loadEvidence("IMPORTANCE"); + EvidenceRow sample = importance.values().iterator().next(); + assertTrue(hasImportance(importance, sample.sourceSoc(), sample.elementId(), sample.dataValue()), + "Known IMPORTANCE evidence row should be present"); + assertFalse(hasImportance(importance, "99-9999.00", sample.elementId(), sample.dataValue()), + "Missing SOC must not match IMPORTANCE evidence"); + assertFalse(hasImportance(importance, sample.sourceSoc(), "9.Z.9.z", sample.dataValue()), + "Missing Element ID must not match IMPORTANCE evidence"); + assertFalse(hasImportance(importance, sample.sourceSoc(), sample.elementId(), + sample.dataValue().add(new BigDecimal("0.01"))), + "Tampered Data Value must not match IMPORTANCE evidence"); + } + + @Test + void skillsHaveChineseNamesAliasesAndTraceableSourceVersions() { + assertEquals(0L, count( + "SELECT COUNT(*) FROM skills WHERE name_zh IS NULL OR TRIM(name_zh) = ''")); + assertEquals(0L, count( + "SELECT COUNT(*) FROM skills WHERE source_code IS NULL OR TRIM(source_code) = ''")); + assertEquals(0L, count( + "SELECT COUNT(*) FROM skills s " + + "WHERE NOT EXISTS (SELECT 1 FROM skill_aliases a WHERE a.skill_id = s.id)")); + assertEquals(0L, count( + "SELECT COUNT(*) FROM (" + + "SELECT normalized_alias FROM skill_aliases " + + "GROUP BY normalized_alias HAVING COUNT(*) > 1" + + ") duplicates")); + + assertEquals(0L, count( + "SELECT COUNT(*) FROM skills s " + + "JOIN skill_source_versions sv ON sv.id = s.source_version_id " + + "WHERE (s.source_type = 'ESCO' AND sv.source_name <> 'ESCO') " + + "OR (s.source_type = 'O*NET' AND sv.source_name <> 'O*NET')")); + } + + @Test + void sourceVersionsMatchTheApprovedManifest() { + assertEquals(1L, count( + "SELECT COUNT(*) FROM skill_source_versions " + + "WHERE source_name = 'ESCO' " + + "AND source_version = '1.2.1 (Release v5.7.0, Last update 10/12/2025)' " + + "AND license = 'Commission Decision 2011/833/EU' " + + "AND attribution_url = 'https://esco.ec.europa.eu/en/use-esco/download'")); + assertEquals(1L, count( + "SELECT COUNT(*) FROM skill_source_versions " + + "WHERE source_name = 'O*NET' " + + "AND source_version = '31.0 Database' " + + "AND license = 'CC BY 4.0' " + + "AND attribution_url = 'https://www.onetcenter.org/database.html'")); + assertEquals(2L, count("SELECT COUNT(*) FROM skill_source_versions")); + } + + @Test + void eachIndustryHasAtLeastOneRepresentativeSkill() { + assertTrue(hasSkillInIndustry("INTERNET", "JAVASCRIPT")); + assertTrue(hasSkillInIndustry("FINANCE", "FINANCIAL_ANALYSIS")); + assertTrue(hasSkillInIndustry("EDUCATION", "INSTRUCTIONAL_DESIGN")); + } + + + private static final String EXPECTED_MAPPINGS_RESOURCE = + "skill-cloud/skill-cloud-expected-mappings.csv"; + private static final String EXPECTED_MAPPINGS_HEADER = + "record_type,entity_key,expected_source_type,expected_source_code," + + "expected_element_id,expected_importance,expected_is_proxy"; + + @Test + void expectedMappingFileMatchesSeedAndSourceEvidenceExactly() { + Map expected = loadExpectedMappings(); + Map softwareEvidence = loadEvidence("SOFTWARE"); + Map importanceEvidence = loadEvidence("IMPORTANCE"); + + assertExpectedMappingCounts(expected); + verifyExpectedMappings(expected, softwareEvidence, importanceEvidence); + assertMappingVerifierRejectsTampering(expected); + } + + private Map loadExpectedMappings() { + Map mappings = new LinkedHashMap<>(); + try (InputStream input = new ClassPathResource(EXPECTED_MAPPINGS_RESOURCE).getInputStream(); + BufferedReader reader = new BufferedReader( + new InputStreamReader(input, StandardCharsets.UTF_8))) { + assertEquals(EXPECTED_MAPPINGS_HEADER, reader.readLine(), + "Unexpected expected-mappings header in classpath:" + EXPECTED_MAPPINGS_RESOURCE); + + String line; + int lineNumber = 1; + while ((line = reader.readLine()) != null) { + lineNumber++; + if (line.isBlank()) { + continue; + } + List values = parseCsvLine(line); + assertEquals(7, values.size(), + "Malformed expected-mappings line " + lineNumber + ": " + line); + String recordType = values.get(0); + String entityKey = values.get(1); + if (recordType.isBlank() || entityKey.isBlank()) { + throw expectedMappingFailure(recordType, entityKey, "row", + "non-empty record_type and entity_key", line); + } + ExpectedMapping mapping = new ExpectedMapping( + recordType, + entityKey, + emptyToNull(values.get(2)), + emptyToNull(values.get(3)), + emptyToNull(values.get(4)), + emptyToBigDecimal(values.get(5), recordType, entityKey), + emptyToInteger(values.get(6), recordType, entityKey)); + String key = expectedKey(recordType, entityKey); + ExpectedMapping previous = mappings.putIfAbsent(key, mapping); + if (previous != null) { + throw expectedMappingFailure(recordType, entityKey, "duplicate", + previous, mapping); + } + } + } catch (AssertionError error) { + throw error; + } catch (Exception exception) { + throw new AssertionError( + "Unable to load expected mappings from classpath:" + EXPECTED_MAPPINGS_RESOURCE, + exception); + } + return mappings; + } + + private void assertExpectedMappingCounts(Map mappings) { + Map expectedCounts = Map.of( + "OCCUPATION_SOURCE", 10, + "SKILL_SOURCE", 72, + "RELATION", 139); + Map actualCounts = new LinkedHashMap<>(); + expectedCounts.keySet().forEach(recordType -> actualCounts.put(recordType, 0)); + for (ExpectedMapping mapping : mappings.values()) { + actualCounts.merge(mapping.recordType(), 1, Integer::sum); + } + for (Map.Entry entry : expectedCounts.entrySet()) { + Integer actual = actualCounts.get(entry.getKey()); + if (!entry.getValue().equals(actual)) { + throw expectedMappingFailure(entry.getKey(), "", "count", + entry.getValue(), actual); + } + } + for (Map.Entry entry : actualCounts.entrySet()) { + if (!expectedCounts.containsKey(entry.getKey())) { + throw expectedMappingFailure(entry.getKey(), "", "record_type", + expectedCounts.keySet(), actualCounts.keySet()); + } + } + } + + private void verifyExpectedMappings( + Map expected, + Map softwareEvidence, + Map importanceEvidence) { + Map actual = loadActualMappings(); + assertSameMappingKeys(expected, actual); + + for (ExpectedMapping expectedMapping : expected.values()) { + assertValidExpectedMapping(expectedMapping); + ActualMapping actualMapping = actual.get(expectedKey( + expectedMapping.recordType(), expectedMapping.entityKey())); + if (actualMapping == null) { + throw expectedMappingFailure(expectedMapping.recordType(), expectedMapping.entityKey(), + "entity", "present", "missing"); + } + assertMappingMatches(expectedMapping, actualMapping); + } + + verifySkillEvidence(expected, softwareEvidence); + verifyRelationEvidence(expected, importanceEvidence); + } + + private void assertMappingVerifierRejectsTampering(Map expected) { + List onetSkills = expected.values().stream() + .filter(mapping -> "SKILL_SOURCE".equals(mapping.recordType())) + .filter(mapping -> "O*NET".equals(mapping.sourceType())) + .toList(); + if (onetSkills.size() < 2) { + throw new AssertionError("Falsifiability self-check requires at least two O*NET skills"); + } + ExpectedMapping firstSkill = onetSkills.get(0); + ExpectedMapping secondSkill = onetSkills.get(1); + ExpectedMapping swappedSource = new ExpectedMapping( + firstSkill.recordType(), firstSkill.entityKey(), firstSkill.sourceType(), + secondSkill.sourceCode(), firstSkill.elementId(), + firstSkill.importance(), firstSkill.isProxy()); + assertThrows(AssertionError.class, + () -> assertMappingMatches(swappedSource, actualFrom(firstSkill)), + "Swapping source_code must fail the generic mapping check"); + + ExpectedMapping weightedRelation = expected.values().stream() + .filter(mapping -> "RELATION".equals(mapping.recordType())) + .filter(mapping -> mapping.importance() != null) + .filter(mapping -> mapping.isProxy() != null) + .findFirst() + .orElseThrow(() -> new AssertionError("No weighted relation for self-check")); + ExpectedMapping toggledProxy = new ExpectedMapping( + weightedRelation.recordType(), weightedRelation.entityKey(), weightedRelation.sourceType(), + weightedRelation.sourceCode(), weightedRelation.elementId(), + weightedRelation.importance(), weightedRelation.isProxy() == 0 ? 1 : 0); + assertThrows(AssertionError.class, + () -> assertMappingMatches(toggledProxy, actualFrom(weightedRelation)), + "Toggling expected_is_proxy must fail the generic mapping check"); + + ExpectedMapping tamperedImportance = new ExpectedMapping( + weightedRelation.recordType(), weightedRelation.entityKey(), weightedRelation.sourceType(), + weightedRelation.sourceCode(), weightedRelation.elementId(), + weightedRelation.importance().add(new BigDecimal("0.01")), + weightedRelation.isProxy()); + assertThrows(AssertionError.class, + () -> assertMappingMatches(tamperedImportance, actualFrom(weightedRelation)), + "Tampering expected_importance must fail the generic mapping check"); + } + + private Map loadActualMappings() { + Map mappings = new LinkedHashMap<>(); + + List> occupationRows = jdbcTemplate.queryForList( + "SELECT o.code AS entity_key, sv.source_name AS source_type, o.source_code " + + "FROM skill_occupations o " + + "JOIN skill_source_versions sv ON sv.id = o.source_version_id"); + for (Map row : occupationRows) { + String entityKey = (String) row.get("entity_key"); + putActual(mappings, "OCCUPATION_SOURCE", entityKey, new ActualMapping( + (String) row.get("source_type"), + (String) row.get("source_code"), + null, + null, + null)); + } + + List> skillRows = jdbcTemplate.queryForList( + "SELECT code AS entity_key, source_type, source_code FROM skills"); + for (Map row : skillRows) { + String entityKey = (String) row.get("entity_key"); + putActual(mappings, "SKILL_SOURCE", entityKey, new ActualMapping( + (String) row.get("source_type"), + (String) row.get("source_code"), + null, + null, + null)); + } + + List> relationRows = jdbcTemplate.queryForList( + "SELECT o.code AS occupation_code, s.code AS skill_code, " + + "sv.source_name AS source_type, os.importance_source_code, " + + "os.importance_element_id, os.importance, os.importance_is_proxy " + + "FROM occupation_skills os " + + "JOIN skill_occupations o ON o.id = os.occupation_id " + + "JOIN skills s ON s.id = os.skill_id " + + "JOIN skill_source_versions sv ON sv.id = os.source_version_id"); + for (Map row : relationRows) { + String entityKey = row.get("occupation_code") + "|" + row.get("skill_code"); + putActual(mappings, "RELATION", entityKey, new ActualMapping( + (String) row.get("source_type"), + (String) row.get("importance_source_code"), + (String) row.get("importance_element_id"), + (BigDecimal) row.get("importance"), + toInteger(row.get("importance_is_proxy")))); + } + return mappings; + } + + private void putActual(Map mappings, String recordType, + String entityKey, ActualMapping mapping) { + String key = expectedKey(recordType, entityKey); + ActualMapping previous = mappings.putIfAbsent(key, mapping); + if (previous != null) { + throw expectedMappingFailure(recordType, entityKey, "duplicate", previous, mapping); + } + } + + private Integer toInteger(Object value) { + if (value == null) { + return null; + } + if (value instanceof Boolean booleanValue) { + return booleanValue ? 1 : 0; + } + return ((Number) value).intValue(); + } + + private void assertSameMappingKeys( + Map expected, Map actual) { + Set expectedKeys = new TreeSet<>(expected.keySet()); + Set actualKeys = new TreeSet<>(actual.keySet()); + if (!expectedKeys.equals(actualKeys)) { + Set missing = new TreeSet<>(expectedKeys); + missing.removeAll(actualKeys); + if (!missing.isEmpty()) { + String key = missing.iterator().next(); + throw expectedMappingFailure(keyPart(key, 0), keyPart(key, 1), + "entity", "present", "missing"); + } + Set extra = new TreeSet<>(actualKeys); + extra.removeAll(expectedKeys); + String key = extra.iterator().next(); + throw expectedMappingFailure(keyPart(key, 0), keyPart(key, 1), + "entity", "absent", "present"); + } + } + + private String keyPart(String key, int index) { + return key.split("\\u0000", -1)[index]; + } + + private void assertValidExpectedMapping(ExpectedMapping mapping) { + String recordType = mapping.recordType(); + String entityKey = mapping.entityKey(); + if ("OCCUPATION_SOURCE".equals(recordType)) { + assertExpectedField(mapping, "expected_source_type", "ESCO", mapping.sourceType()); + assertExpectedNonNull(mapping, "expected_source_code", mapping.sourceCode()); + assertExpectedField(mapping, "expected_element_id", null, mapping.elementId()); + assertExpectedField(mapping, "expected_importance", null, mapping.importance()); + assertExpectedField(mapping, "expected_is_proxy", null, mapping.isProxy()); + return; + } + if ("SKILL_SOURCE".equals(recordType)) { + if (!"ESCO".equals(mapping.sourceType()) && !"O*NET".equals(mapping.sourceType())) { + throw expectedMappingFailure(recordType, entityKey, "expected_source_type", + "ESCO or O*NET", mapping.sourceType()); + } + assertExpectedNonNull(mapping, "expected_source_code", mapping.sourceCode()); + assertExpectedField(mapping, "expected_element_id", null, mapping.elementId()); + assertExpectedField(mapping, "expected_importance", null, mapping.importance()); + assertExpectedField(mapping, "expected_is_proxy", null, mapping.isProxy()); + return; + } + if ("RELATION".equals(recordType)) { + assertExpectedField(mapping, "expected_source_type", "O*NET", mapping.sourceType()); + if (mapping.isProxy() == null) { + throw expectedMappingFailure(recordType, entityKey, "expected_is_proxy", + "0 or 1", null); + } + boolean hasProvenance = mapping.sourceCode() != null + || mapping.elementId() != null + || mapping.importance() != null; + boolean completeProvenance = mapping.sourceCode() != null + && mapping.elementId() != null + && mapping.importance() != null; + if (hasProvenance != completeProvenance) { + throw expectedMappingFailure(recordType, entityKey, "importance_provenance", + "all empty or all present", + mapping.sourceCode() + "|" + mapping.elementId() + "|" + mapping.importance()); + } + if (!completeProvenance && mapping.isProxy() != 0) { + throw expectedMappingFailure(recordType, entityKey, "expected_is_proxy", + 0, mapping.isProxy()); + } + return; + } + throw expectedMappingFailure(recordType, entityKey, "record_type", + "OCCUPATION_SOURCE|SKILL_SOURCE|RELATION", recordType); + } + + private void assertExpectedNonNull( + ExpectedMapping mapping, String field, Object actual) { + if (actual == null) { + throw expectedMappingFailure( + mapping.recordType(), mapping.entityKey(), field, "non-empty", null); + } + } + + private void assertExpectedField( + ExpectedMapping mapping, String field, Object expected, Object actual) { + if (expected == null ? actual != null : !expected.equals(actual)) { + throw expectedMappingFailure( + mapping.recordType(), mapping.entityKey(), field, expected, actual); + } + } + + private void assertMappingMatches(ExpectedMapping expected, ActualMapping actual) { + assertMappingField(expected.recordType(), expected.entityKey(), + "source_type", expected.sourceType(), actual.sourceType()); + assertMappingField(expected.recordType(), expected.entityKey(), + "source_code", expected.sourceCode(), actual.sourceCode()); + assertMappingField(expected.recordType(), expected.entityKey(), + "element_id", expected.elementId(), actual.elementId()); + + if (expected.importance() == null) { + assertMappingField(expected.recordType(), expected.entityKey(), + "importance", null, actual.importance()); + } else if (actual.importance() == null + || actual.importance().subtract(expected.importance()).abs() + .compareTo(IMPORTANCE_TOLERANCE) > 0) { + throw expectedMappingFailure(expected.recordType(), expected.entityKey(), + "importance", expected.importance(), actual.importance()); + } + + assertMappingField(expected.recordType(), expected.entityKey(), + "importance_is_proxy", expected.isProxy(), actual.isProxy()); + } + + private void assertMappingField( + String recordType, String entityKey, String field, Object expected, Object actual) { + if (expected == null ? actual != null : !expected.equals(actual)) { + throw expectedMappingFailure(recordType, entityKey, field, expected, actual); + } + } + + private void verifySkillEvidence( + Map expected, Map softwareEvidence) { + Set expectedSoftwareKeys = new TreeSet<>(); + for (ExpectedMapping mapping : expected.values()) { + if (!"SKILL_SOURCE".equals(mapping.recordType()) || !"O*NET".equals(mapping.sourceType())) { + continue; + } + String[] parts = mapping.sourceCode().split("\\|", -1); + if (parts.length != 3 + || !ONET_SKILL_SOURCE_PATTERN.matcher(mapping.sourceCode()).matches()) { + throw expectedMappingFailure(mapping.recordType(), mapping.entityKey(), + "expected_source_code", "SOC|Workplace Example|Element ID", mapping.sourceCode()); + } + String key = softwareKey(parts[0], parts[1], parts[2]); + EvidenceRow evidence = softwareEvidence.get(key); + if (evidence == null) { + throw expectedMappingFailure(mapping.recordType(), mapping.entityKey(), + "expected_source_code", "present in onet-31-reference.csv", "missing"); + } + if (!"software_skills.csv".equals(evidence.sourceFile())) { + throw expectedMappingFailure(mapping.recordType(), mapping.entityKey(), + "source_file", "software_skills.csv", evidence.sourceFile()); + } + if (evidence.dataValue() != null) { + throw expectedMappingFailure(mapping.recordType(), mapping.entityKey(), + "data_value", null, evidence.dataValue()); + } + expectedSoftwareKeys.add(key); + } + assertEvidenceKeySet("SOFTWARE", expectedSoftwareKeys, softwareEvidence.keySet()); + } + + private void verifyRelationEvidence( + Map expected, Map importanceEvidence) { + Set expectedImportanceKeys = new TreeSet<>(); + for (ExpectedMapping mapping : expected.values()) { + if (!"RELATION".equals(mapping.recordType()) || mapping.importance() == null) { + continue; + } + String key = importanceKey(mapping.sourceCode(), mapping.elementId()); + EvidenceRow evidence = importanceEvidence.get(key); + if (evidence == null) { + throw expectedMappingFailure(mapping.recordType(), mapping.entityKey(), + "importance_source_code", "present in onet-31-reference.csv", "missing"); + } + if (evidence.dataValue() == null + || evidence.dataValue().subtract(mapping.importance()).abs() + .compareTo(IMPORTANCE_TOLERANCE) > 0) { + throw expectedMappingFailure(mapping.recordType(), mapping.entityKey(), + "expected_importance", mapping.importance(), evidence.dataValue()); + } + expectedImportanceKeys.add(key); + } + assertEvidenceKeySet("IMPORTANCE", expectedImportanceKeys, importanceEvidence.keySet()); + } + + private void assertEvidenceKeySet( + String recordType, Set expectedKeys, Set actualKeys) { + Set missing = new TreeSet<>(expectedKeys); + missing.removeAll(actualKeys); + if (!missing.isEmpty()) { + throw expectedMappingFailure(recordType, missing.iterator().next(), + "evidence_key", "present", "missing"); + } + Set extra = new TreeSet<>(actualKeys); + extra.removeAll(expectedKeys); + if (!extra.isEmpty()) { + throw expectedMappingFailure(recordType, extra.iterator().next(), + "evidence_key", "absent", "present"); + } + } + + private ActualMapping actualFrom(ExpectedMapping mapping) { + return new ActualMapping( + mapping.sourceType(), + mapping.sourceCode(), + mapping.elementId(), + mapping.importance(), + mapping.isProxy()); + } + + private String expectedKey(String recordType, String entityKey) { + return recordType + '\u0000' + entityKey; + } + + private String emptyToNull(String value) { + return value == null || value.isBlank() ? null : value; + } + + private BigDecimal emptyToBigDecimal( + String value, String recordType, String entityKey) { + String normalized = emptyToNull(value); + if (normalized == null) { + return null; + } + try { + return new BigDecimal(normalized); + } catch (NumberFormatException exception) { + throw expectedMappingFailure(recordType, entityKey, "expected_importance", + "decimal or empty", value); + } + } + + private Integer emptyToInteger(String value, String recordType, String entityKey) { + String normalized = emptyToNull(value); + if (normalized == null) { + return null; + } + try { + int parsed = Integer.parseInt(normalized); + if (parsed != 0 && parsed != 1) { + throw new NumberFormatException("proxy must be 0 or 1"); + } + return parsed; + } catch (NumberFormatException exception) { + throw expectedMappingFailure(recordType, entityKey, "expected_is_proxy", + "0 or 1", value); + } + } + + private AssertionError expectedMappingFailure( + String recordType, String entityKey, String field, Object expected, Object actual) { + return new AssertionError( + "Expected mapping mismatch: record_type=" + recordType + + ", entity_key=" + entityKey + + ", field=" + field + + ", expected=" + expected + + ", actual=" + actual); + } + + private record ExpectedMapping( + String recordType, + String entityKey, + String sourceType, + String sourceCode, + String elementId, + BigDecimal importance, + Integer isProxy + ) { + } + + private record ActualMapping( + String sourceType, + String sourceCode, + String elementId, + BigDecimal importance, + Integer isProxy + ) { + } + + private Map loadEvidence(String evidenceType) { + Map rows = new LinkedHashMap<>(); + try (InputStream input = new ClassPathResource(EVIDENCE_RESOURCE).getInputStream(); + BufferedReader reader = new BufferedReader( + new InputStreamReader(input, StandardCharsets.UTF_8))) { + assertEquals(EVIDENCE_HEADER, reader.readLine(), + "Unexpected evidence snapshot header in classpath:" + EVIDENCE_RESOURCE); + + String line; + int lineNumber = 1; + while ((line = reader.readLine()) != null) { + lineNumber++; + if (line.isBlank()) { + continue; + } + List values = parseCsvLine(line); + assertEquals(6, values.size(), + "Malformed evidence snapshot line " + lineNumber + ": " + line); + if (!evidenceType.equals(values.get(0))) { + continue; + } + + String sourceSoc = values.get(1); + String elementId = values.get(2); + BigDecimal dataValue = values.get(3).isEmpty() + ? null : new BigDecimal(values.get(3)); + String workplaceExample = values.get(4); + String sourceFile = values.get(5); + + if ("SOFTWARE".equals(evidenceType)) { + assertEquals("software_skills.csv", sourceFile, + "Unexpected SOFTWARE source file at line " + lineNumber); + } else { + assertTrue(IMPORTANCE_SOURCE_FILES.contains(sourceFile), + "Unexpected IMPORTANCE source file at line " + lineNumber + + ": sourceFile=" + sourceFile); + } + + String key = "SOFTWARE".equals(evidenceType) + ? softwareKey(sourceSoc, workplaceExample, elementId) + : importanceKey(sourceSoc, elementId); + EvidenceRow evidence = new EvidenceRow( + evidenceType, sourceSoc, elementId, dataValue, workplaceExample, sourceFile); + EvidenceRow previous = rows.putIfAbsent(key, evidence); + if (previous != null) { + throw new AssertionError("Duplicate evidence row at line " + lineNumber + + ": evidenceType=" + evidenceType + + ", key=" + key + ", previous=" + previous + ", actual=" + evidence); + } + } + } catch (Exception exception) { + throw new AssertionError( + "Unable to load evidence snapshot from classpath:" + EVIDENCE_RESOURCE, exception); + } + return rows; + } + + private List parseCsvLine(String line) { + List values = new ArrayList<>(); + StringBuilder current = new StringBuilder(); + boolean quoted = false; + for (int index = 0; index < line.length(); index++) { + char character = line.charAt(index); + if (character == '"') { + if (quoted && index + 1 < line.length() && line.charAt(index + 1) == '"') { + current.append('"'); + index++; + } else { + quoted = !quoted; + } + } else if (character == ',' && !quoted) { + values.add(current.toString()); + current.setLength(0); + } else { + current.append(character); + } + } + assertFalse(quoted, "Unclosed quoted field in evidence snapshot line: " + line); + values.add(current.toString()); + return values; + } + + private boolean hasSoftware(Map rows, + String sourceSoc, String workplaceExample, String elementId) { + return rows.containsKey(softwareKey(sourceSoc, workplaceExample, elementId)); + } + + private boolean hasImportance(Map rows, + String sourceSoc, String elementId, BigDecimal expectedValue) { + if (expectedValue == null) { + return false; + } + EvidenceRow evidence = rows.get(importanceKey(sourceSoc, elementId)); + return evidence != null + && evidence.dataValue() != null + && evidence.dataValue().subtract(expectedValue).abs() + .compareTo(IMPORTANCE_TOLERANCE) <= 0; + } + + private String softwareKey(String sourceSoc, String workplaceExample, String elementId) { + return sourceSoc + '\u0000' + workplaceExample + '\u0000' + elementId; + } + + private String importanceKey(String sourceSoc, String elementId) { + return sourceSoc + '\u0000' + elementId; + } + + private void assertOnetSkillSource(String skillCode, String expectedSource) { + assertEquals(1L, count( + "SELECT COUNT(*) FROM skills " + + "WHERE code = ? AND source_type = 'O*NET' AND source_code = ?", + skillCode, expectedSource), "Unexpected O*NET source for " + skillCode); + } + + private void assertImportance(String occupationCode, String skillCode, + String sourceCode, String elementId, String expectedValue) { + Map row = jdbcTemplate.queryForMap( + "SELECT os.importance, os.importance_source_code, os.importance_element_id, " + + "os.importance_is_proxy " + + "FROM occupation_skills os " + + "JOIN skill_occupations o ON o.id = os.occupation_id " + + "JOIN skills s ON s.id = os.skill_id " + + "WHERE o.code = ? AND s.code = ?", + occupationCode, skillCode); + assertEquals(new BigDecimal(expectedValue), ((BigDecimal) row.get("importance")).setScale(2)); + assertEquals(sourceCode, row.get("importance_source_code")); + assertEquals(elementId, row.get("importance_element_id")); + assertEquals(1, toInteger(row.get("importance_is_proxy"))); + } + + private void assertNullImportance(String occupationCode, String skillCode) { + Map row = jdbcTemplate.queryForMap( + "SELECT os.importance, os.importance_source_code, os.importance_element_id, " + + "os.importance_is_proxy " + + "FROM occupation_skills os " + + "JOIN skill_occupations o ON o.id = os.occupation_id " + + "JOIN skills s ON s.id = os.skill_id " + + "WHERE o.code = ? AND s.code = ?", + occupationCode, skillCode); + assertNull(row.get("importance")); + assertNull(row.get("importance_source_code")); + assertNull(row.get("importance_element_id")); + assertEquals(0, toInteger(row.get("importance_is_proxy"))); + } + + private boolean hasSkillInIndustry(String industryCode, String skillCode) { + return count( + "SELECT COUNT(*) FROM occupation_skills os " + + "JOIN skill_occupations o ON o.id = os.occupation_id " + + "JOIN skill_industries i ON i.id = o.industry_id " + + "JOIN skills s ON s.id = os.skill_id " + + "WHERE i.code = ? AND s.code = ?", + industryCode, skillCode) > 0L; + } + + private long count(String sql, Object... args) { + Long result = jdbcTemplate.queryForObject(sql, Long.class, args); + assertNotNull(result); + return result; + } + + private long number(Object value) { + assertNotNull(value); + return ((Number) value).longValue(); + } + + private record EvidenceRow( + String evidenceType, + String sourceSoc, + String elementId, + BigDecimal dataValue, + String workplaceExample, + String sourceFile + ) { + } +} diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillLearningPathServiceTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillLearningPathServiceTest.java new file mode 100644 index 0000000..5143d58 --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillLearningPathServiceTest.java @@ -0,0 +1,416 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.skill.domain.SkillCategory; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillLearningPathItem; +import com.mydreamjob.skill.dto.SkillLearningPathResponse; +import com.mydreamjob.skill.dto.SkillMatchResponse; +import com.mydreamjob.skill.repository.SkillCategoryRepository; +import org.hibernate.boot.model.relational.Namespace; +import org.hibernate.boot.model.relational.Sequence; +import org.hibernate.mapping.Table; +import org.hibernate.tool.schema.spi.SchemaFilter; +import org.hibernate.tool.schema.spi.SchemaFilterProvider; +import org.junit.jupiter.api.Nested; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; +import org.springframework.transaction.annotation.Transactional; + +import java.math.BigDecimal; +import java.util.List; +import java.util.Locale; +import java.util.Optional; +import java.util.Set; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; +import static org.mockito.Mockito.verifyNoMoreInteractions; +import static org.mockito.Mockito.when; + +@SpringBootTest +@TestPropertySource(properties = { + "spring.datasource.url=jdbc:h2:mem:integrationdb_skill_learning_path;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE", + "spring.jpa.hibernate.ddl-auto=validate", + "spring.jpa.properties.hibernate.hbm2ddl.schema_filter_provider=com.mydreamjob.skill.SkillLearningPathServiceTest$SkillLearningPathSchemaFilterProvider" +}) +@ActiveProfiles("integration-test") +@Transactional +class SkillLearningPathServiceTest { + + private static final Long USER_ID = 7001L; + private static final Long INDUSTRY_ID = 1L; + private static final Long OCCUPATION_ID = 1L; + private static final Long CATEGORY_ID = 101L; + private static final Long EDUCATION_INDUSTRY_ID = 3L; + private static final Long INSTRUCTIONAL_DESIGNER_OCCUPATION_ID = 9L; + + @Autowired + private SkillLearningPathService learningPathService; + + @Autowired + private SkillMatchService matchService; + + @MockBean + private RedisTemplate redisTemplate; + + @Test + void v36LearningPathReusesMatchMissingSkillsWithoutReordering() { + SkillMatchResponse matchResponse = matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + SkillLearningPathResponse response = learningPathService.generate( + USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + + assertThat(matchResponse.missingSkills()).hasSize(10); + assertThat(response.total()).isEqualTo(matchResponse.missingSkills().size()); + assertThat(response.items()) + .extracting(SkillLearningPathItem::skillId) + .containsExactlyElementsOf(matchResponse.missingSkills().stream() + .map(SkillCloudItem::skillId) + .toList()); + assertThat(response.items()) + .extracting(SkillLearningPathItem::priority) + .containsExactly(1, 2, 3, 4, 5, 6, 7, 8, 9, 10); + assertThat(response.items()).allSatisfy(item -> { + assertThat(item.skillName()).isNotBlank(); + assertThat(item.categoryId()).isNotNull(); + assertThat(item.categoryName()).isNotBlank(); + assertThat(item.suggestion()).contains(item.skillName()); + assertNoForbiddenContent(item.suggestion()); + }); + } + + @Test + void v36LearningPathPreservesMatchOrderAcrossHeatGroupsWhenNameOrderDiffers() { + SkillMatchResponse matchResponse = matchService.match( + USER_ID, EDUCATION_INDUSTRY_ID, INSTRUCTIONAL_DESIGNER_OCCUPATION_ID, null); + SkillLearningPathResponse response = learningPathService.generate( + USER_ID, EDUCATION_INDUSTRY_ID, INSTRUCTIONAL_DESIGNER_OCCUPATION_ID, null); + + List missingSkills = matchResponse.missingSkills(); + List missingNames = missingSkills.stream() + .map(SkillCloudItem::nameZh) + .toList(); + List missingIds = missingSkills.stream() + .map(SkillCloudItem::skillId) + .toList(); + List distinctHeats = missingSkills.stream() + .map(SkillCloudItem::heat) + .distinct() + .toList(); + + assertThat(missingSkills).hasSize(10); + assertThat(distinctHeats.size()).isGreaterThan(1); + assertThat(missingNames).isNotEqualTo(missingNames.stream().sorted().toList()); + assertThat(missingIds).isNotEqualTo(missingIds.stream().sorted().toList()); + assertThat(response.total()).isEqualTo(missingSkills.size()); + assertThat(response.items()) + .extracting(SkillLearningPathItem::skillId) + .containsExactlyElementsOf(missingIds); + assertThat(response.items()) + .extracting(SkillLearningPathItem::skillName) + .containsExactlyElementsOf(missingNames); + assertThat(response.items()) + .extracting(SkillLearningPathItem::priority) + .containsExactly(1, 2, 3, 4, 5, 6, 7, 8, 9, 10); + } + + @Nested + class TemplateAndOrderingTests { + + private final SkillMatchService matchService = mock(SkillMatchService.class); + private final SkillCategoryRepository categoryRepository = mock(SkillCategoryRepository.class); + private final SkillLearningPathService service = + new SkillLearningPathService(matchService, categoryRepository); + + @Test + void usesOnlyMissingSkillsAndPreservesMissingOrderWithContiguousPriorities() { + SkillCloudItem matched = skill(11L, "Matched", "Matched", CATEGORY_ID, + "编程语言", new BigDecimal("9.00")); + List missing = List.of( + skill(30L, "Skill Z", "Skill Z", 101L, "编程语言", new BigDecimal("1.00")), + skill(10L, "Skill A", "Skill A", 102L, "框架", new BigDecimal("10.00")), + skill(20L, "Skill M", "Skill M", 103L, "工具", new BigDecimal("5.00")) + ); + List missingNames = missing.stream() + .map(SkillCloudItem::nameZh) + .toList(); + List missingIds = missing.stream() + .map(SkillCloudItem::skillId) + .toList(); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(matched), missing)); + stubCategory(101L, "PROGRAMMING_LANGUAGE"); + stubCategory(102L, "FRAMEWORK"); + stubCategory(103L, "TOOL"); + + SkillLearningPathResponse response = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + + assertThat(response.total()).isEqualTo(3); + assertThat(missingNames).isNotEqualTo(missingNames.stream().sorted().toList()); + assertThat(missingIds).isNotEqualTo(missingIds.stream().sorted().toList()); + assertThat(response.items()) + .extracting(SkillLearningPathItem::skillId) + .containsExactly(30L, 10L, 20L); + assertThat(response.items()) + .extracting(SkillLearningPathItem::priority) + .containsExactly(1, 2, 3); + assertThat(response.items()) + .extracting(SkillLearningPathItem::skillName) + .containsExactly("Skill Z", "Skill A", "Skill M"); + assertThat(response.items()) + .noneMatch(item -> item.skillId().equals(matched.skillId())); + verify(matchService).match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + verifyNoMoreInteractions(matchService); + verify(categoryRepository).findByIdAndOccupationId(101L, OCCUPATION_ID); + verify(categoryRepository).findByIdAndOccupationId(102L, OCCUPATION_ID); + verify(categoryRepository).findByIdAndOccupationId(103L, OCCUPATION_ID); + verifyNoMoreInteractions(categoryRepository); + } + + @Test + void generatesProgrammingLanguageSuggestion() { + SkillCloudItem missing = skill(41L, "Java", "Java", CATEGORY_ID, + "编程语言", BigDecimal.ONE); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(), List.of(missing))); + stubCategory(CATEGORY_ID, "PROGRAMMING_LANGUAGE"); + + SkillLearningPathItem item = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null) + .items().get(0); + + assertThat(item.suggestion()).isEqualTo( + "先掌握Java的基础语法和常见数据结构，再完成至少一个编码练习。"); + assertNoForbiddenContent(item.suggestion()); + } + + @Test + void generatesFrameworkSuggestion() { + SkillCloudItem missing = skill(42L, "Spring Boot", "Spring Boot", CATEGORY_ID, + "框架", BigDecimal.ONE); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(), List.of(missing))); + stubCategory(CATEGORY_ID, "FRAMEWORK"); + + SkillLearningPathItem item = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null) + .items().get(0); + + assertThat(item.suggestion()).isEqualTo( + "先理解Spring Boot的核心概念与生命周期，再通过一个小型项目练习集成。"); + assertNoForbiddenContent(item.suggestion()); + } + + @Test + void generatesToolSuggestion() { + SkillCloudItem missing = skill(43L, "Docker", "Docker", CATEGORY_ID, + "工具", BigDecimal.ONE); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(), List.of(missing))); + stubCategory(CATEGORY_ID, "TOOL"); + + SkillLearningPathItem item = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null) + .items().get(0); + + assertThat(item.suggestion()).isEqualTo( + "先完成Docker的安装与基础配置，再结合现有工程完成一次实际操作。"); + assertNoForbiddenContent(item.suggestion()); + } + + @Test + void generatesGeneralSuggestion() { + SkillCloudItem missing = skill(44L, "沟通能力", "Communication", CATEGORY_ID, + "通用能力", BigDecimal.ONE); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(), List.of(missing))); + stubCategory(CATEGORY_ID, "GENERAL"); + + SkillLearningPathItem item = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null) + .items().get(0); + + assertThat(item.suggestion()).isEqualTo( + "先了解沟通能力的核心概念和常见应用场景，再通过一个简短实践巩固。"); + assertNoForbiddenContent(item.suggestion()); + } + + @Test + void fallsBackToGeneralForUnknownNullAndMissingTemplateCodes() { + List missing = List.of( + skill(51L, "Unknown", "Unknown", 101L, "分类 A", BigDecimal.ONE), + skill(52L, "NullCode", "NullCode", 102L, "分类 B", BigDecimal.ONE), + skill(53L, "MissingCategory", "MissingCategory", 103L, "分类 C", BigDecimal.ONE) + ); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(), missing)); + when(categoryRepository.findByIdAndOccupationId(101L, OCCUPATION_ID)) + .thenReturn(Optional.of(category(101L, "UNKNOWN_TEMPLATE"))); + when(categoryRepository.findByIdAndOccupationId(102L, OCCUPATION_ID)) + .thenReturn(Optional.of(category(102L, null))); + when(categoryRepository.findByIdAndOccupationId(103L, OCCUPATION_ID)) + .thenReturn(Optional.empty()); + + SkillLearningPathResponse response = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + + assertThat(response.items()) + .extracting(SkillLearningPathItem::suggestion) + .containsExactly( + "先了解Unknown的核心概念和常见应用场景，再通过一个简短实践巩固。", + "先了解NullCode的核心概念和常见应用场景，再通过一个简短实践巩固。", + "先了解MissingCategory的核心概念和常见应用场景，再通过一个简短实践巩固。" + ); + response.items().forEach(item -> assertNoForbiddenContent(item.suggestion())); + } + + @Test + void returnsEmptyPathAndDoesNotReadCategoriesWhenThereAreNoGaps() { + SkillCloudItem matched = skill(61L, "Java", "Java", CATEGORY_ID, + "编程语言", BigDecimal.ONE); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(matched), List.of())); + + SkillLearningPathResponse response = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + + assertThat(response.total()).isZero(); + assertThat(response.items()).isEmpty(); + verifyNoInteractions(categoryRepository); + } + + @Test + void repeatedCallsProduceIdenticalOrderPrioritiesAndText() { + List missing = List.of( + skill(71L, "Java", "Java", 101L, "编程语言", BigDecimal.ONE), + skill(72L, "Docker", "Docker", 102L, "工具", BigDecimal.ONE) + ); + when(matchService.match(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null)) + .thenReturn(matchResponse(List.of(), missing)); + stubCategory(101L, "PROGRAMMING_LANGUAGE"); + stubCategory(102L, "TOOL"); + + SkillLearningPathResponse first = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + SkillLearningPathResponse second = service.generate(USER_ID, INDUSTRY_ID, OCCUPATION_ID, null); + + assertThat(second).isEqualTo(first); + assertThat(second.items()) + .extracting(SkillLearningPathItem::skillId) + .containsExactly(71L, 72L); + assertThat(second.items()) + .extracting(SkillLearningPathItem::priority) + .containsExactly(1, 2); + assertThat(second.items()) + .extracting(SkillLearningPathItem::suggestion) + .containsExactly( + "先掌握Java的基础语法和常见数据结构，再完成至少一个编码练习。", + "先完成Docker的安装与基础配置，再结合现有工程完成一次实际操作。" + ); + } + + private void stubCategory(Long categoryId, String templateCode) { + when(categoryRepository.findByIdAndOccupationId(categoryId, OCCUPATION_ID)) + .thenReturn(Optional.of(category(categoryId, templateCode))); + } + } + + private static SkillMatchResponse matchResponse(List matched, + List missing) { + int total = matched.size() + missing.size(); + int score = total == 0 ? 0 : (int) Math.round(matched.size() * 100.0 / total); + return new SkillMatchResponse(score, matched.size(), total, matched, missing); + } + + private static SkillCloudItem skill(Long skillId, + String nameZh, + String nameEn, + Long categoryId, + String categoryName, + BigDecimal heat) { + return new SkillCloudItem( + skillId.intValue(), + skillId, + "SKILL_" + skillId, + nameZh, + nameEn, + "ESCO", + "source-" + skillId, + categoryId, + categoryName, + heat, + "15-1254.00", + "2.E.7.h", + false + ); + } + + private static SkillCategory category(Long categoryId, String templateCode) { + return SkillCategory.builder() + .id(categoryId) + .occupationId(OCCUPATION_ID) + .code("CATEGORY_" + categoryId) + .nameZh("分类 " + categoryId) + .suggestionTemplateCode(templateCode) + .sortOrder(categoryId.intValue()) + .build(); + } + + private static void assertNoForbiddenContent(String suggestion) { + assertThat(suggestion) + .doesNotContain("http://", "https://", "课程", "讲师", "价格", "时长", + "分钟", "小时", "完成度", "AI"); + } + + public static class SkillLearningPathSchemaFilterProvider implements SchemaFilterProvider { + + private static final Set VALIDATED_TABLES = Set.of( + "skill_source_versions", + "skill_industries", + "skill_occupations", + "skill_categories", + "skills", + "skill_aliases", + "occupation_skills" + ); + + @Override + public SchemaFilter getCreateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getDropFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getTruncatorFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getMigrateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getValidateFilter() { + return new SchemaFilter() { + @Override + public boolean includeNamespace(Namespace namespace) { + return true; + } + + @Override + public boolean includeTable(Table table) { + return VALIDATED_TABLES.contains(table.getName().toLowerCase(Locale.ROOT)); + } + + @Override + public boolean includeSequence(Sequence sequence) { + return false; + } + }; + } + } +} diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillMatchServiceTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillMatchServiceTest.java new file mode 100644 index 0000000..d50b6e8 --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillMatchServiceTest.java @@ -0,0 +1,482 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.profile.StudentProfile; +import com.mydreamjob.profile.StudentProfileRepository; +import com.mydreamjob.skill.domain.SkillAlias; +import com.mydreamjob.skill.dto.SkillCloudItem; +import com.mydreamjob.skill.dto.SkillCloudNode; +import com.mydreamjob.skill.dto.SkillCloudResponse; +import com.mydreamjob.skill.dto.SkillMatchResponse; +import com.mydreamjob.skill.repository.SkillAliasRepository; +import org.hibernate.boot.model.relational.Namespace; +import org.hibernate.boot.model.relational.Sequence; +import org.hibernate.mapping.Table; +import org.hibernate.tool.schema.spi.SchemaFilter; +import org.hibernate.tool.schema.spi.SchemaFilterProvider; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.extension.ExtendWith; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.boot.test.mock.mockito.SpyBean; +import org.springframework.boot.test.system.CapturedOutput; +import org.springframework.boot.test.system.OutputCaptureExtension; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; +import org.springframework.transaction.annotation.Transactional; + +import java.math.BigDecimal; +import java.util.List; +import java.util.Locale; +import java.util.Optional; +import java.util.Set; +import java.util.stream.IntStream; +import java.util.stream.Stream; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyCollection; +import static org.mockito.Mockito.doReturn; + +@SpringBootTest +@ExtendWith(OutputCaptureExtension.class) +@TestPropertySource(properties = { + "spring.datasource.url=jdbc:h2:mem:integrationdb_skill_match;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE", + "spring.jpa.hibernate.ddl-auto=validate", + "spring.jpa.properties.hibernate.hbm2ddl.schema_filter_provider=com.mydreamjob.skill.SkillMatchServiceTest$SkillMatchSchemaFilterProvider" +}) +@ActiveProfiles("integration-test") +@Transactional +class SkillMatchServiceTest { + + private static final Long USER_ID = 7001L; + + @Autowired + private SkillMatchService service; + + @SpyBean + private SkillCloudQueryService queryService; + + @SpyBean + private StudentProfileRepository profileRepository; + + @SpyBean + private SkillAliasRepository aliasRepository; + + @Autowired + private JdbcTemplate jdbcTemplate; + + @MockBean + private RedisTemplate redisTemplate; + + @Test + void normalizerAppliesUnicodeRulesAndPreservesProgrammingSymbols() { + SkillNameNormalizer normalizer = new SkillNameNormalizer(); + + assertThat(normalizer.normalize(" Ｓｐｒｉｎｇ‐Ｂｏｏｔ. ")).isEqualTo("spring boot"); + assertThat(normalizer.normalize(" JAVA\n\tSCRIPT ")).isEqualTo("java script"); + assertThat(normalizer.normalize("Node.js")).isEqualTo("nodejs"); + assertThat(normalizer.normalize("C++")).isEqualTo("c++"); + assertThat(normalizer.normalize("C#")).isEqualTo("c#"); + } + + @Test + void usesOnlyTopTenWhenMoreThanTenSkillsExist() { + List items = IntStream.rangeClosed(1, 11) + .mapToObj(index -> item((long) index, "Skill " + index, "Skill " + index, "SKILL_" + index)) + .toList(); + stubCloud(cloud(items)); + saveProfile(USER_ID, json(items.stream().map(SkillCloudItem::nameZh).toList())); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.totalCount()).isEqualTo(10); + assertThat(result.matchedCount()).isEqualTo(10); + assertThat(result.matchScore()).isEqualTo(100); + assertThat(result.matchedSkills()) + .extracting(SkillCloudItem::skillId) + .containsExactlyElementsOf(IntStream.rangeClosed(1, 10).mapToLong(Long::valueOf).boxed().toList()); + assertThat(result.missingSkills()).isEmpty(); + } + + @Test + void usesTenAsDenominatorWhenExactlyTenSkillsExist() { + List items = IntStream.rangeClosed(1, 10) + .mapToObj(index -> item((long) index, "Skill " + index, "Skill " + index, "SKILL_" + index)) + .toList(); + stubCloud(cloud(items)); + saveProfile(USER_ID, json(items.stream().limit(5).map(SkillCloudItem::nameZh).toList())); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.totalCount()).isEqualTo(10); + assertThat(result.matchedCount()).isEqualTo(5); + assertThat(result.matchScore()).isEqualTo(50); + assertThat(result.missingSkills()).hasSize(5); + } + + @Test + void usesActualSizeAsDenominatorAndRoundsOneThirdToThirtyThree() { + List items = List.of( + item(1L, "Java", "Java", "JAVA"), + item(2L, "Python", "Python", "PYTHON"), + item(3L, "SQL", "SQL", "SQL") + ); + stubCloud(cloud(items)); + saveProfile(USER_ID, json(List.of("Java"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.totalCount()).isEqualTo(3); + assertThat(result.matchedCount()).isEqualTo(1); + assertThat(result.matchScore()).isEqualTo(33); + assertThat(result.matchedSkills()).extracting(SkillCloudItem::skillId).containsExactly(1L); + assertThat(result.missingSkills()).extracting(SkillCloudItem::skillId).containsExactly(2L, 3L); + } + + @Test + void returnsZeroAndEmptyListsWhenCloudIsEmpty() { + stubCloud(cloud(List.of())); + saveProfile(USER_ID, json(List.of("Java"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.matchScore()).isZero(); + assertThat(result.matchedCount()).isZero(); + assertThat(result.totalCount()).isZero(); + assertThat(result.matchedSkills()).isEmpty(); + assertThat(result.missingSkills()).isEmpty(); + } + + @Test + void treatsMissingProfileAsNoSkills() { + stubCloud(cloud(List.of(item(1L, "Java", "Java", "JAVA")))); + + SkillMatchResponse result = service.match(9999L, 1L, 1L, null); + + assertThat(result.matchScore()).isZero(); + assertThat(result.matchedSkills()).isEmpty(); + assertThat(result.missingSkills()).extracting(SkillCloudItem::skillId).containsExactly(1L); + } + + @Test + void treatsBlankAndEmptyArrayAsNoSkills() { + stubCloud(cloud(List.of(item(1L, "Java", "Java", "JAVA"), item(2L, "SQL", "SQL", "SQL")))); + doReturn(Optional.of(StudentProfile.builder() + .id(100L) + .userId(USER_ID) + .skillsJson(" ") + .build())) + .when(profileRepository).findByUserId(USER_ID); + + SkillMatchResponse blank = service.match(USER_ID, 1L, 1L, null); + + assertThat(blank.matchScore()).isZero(); + assertThat(blank.missingSkills()).extracting(SkillCloudItem::skillId).containsExactly(1L, 2L); + + doReturn(Optional.of(StudentProfile.builder() + .id(101L) + .userId(USER_ID) + .skillsJson("[]") + .build())) + .when(profileRepository).findByUserId(USER_ID); + + SkillMatchResponse empty = service.match(USER_ID, 1L, 1L, null); + + assertThat(empty.matchScore()).isZero(); + assertThat(empty.matchedSkills()).isEmpty(); + } + + @Test + void treatsMalformedJsonAsNoSkillsWithoutLeakingRawJson(CapturedOutput output) { + String malformedJson = "{\"skills\":[\"Java\"]"; + stubCloud(cloud(List.of(item(1L, "Java", "Java", "JAVA")))); + doReturn(Optional.of(StudentProfile.builder() + .id(102L) + .userId(USER_ID) + .skillsJson(malformedJson) + .build())) + .when(profileRepository).findByUserId(USER_ID); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.matchScore()).isZero(); + assertThat(result.matchedSkills()).isEmpty(); + assertThat(output.getAll()).contains("Failed to parse student skills JSON"); + assertThat(output.getAll()).doesNotContain(malformedJson); + } + + @Test + void matchesNameZhNameEnSkillCodeAndAliasAndDeduplicatesStudentSkills() { + List items = List.of( + item(1L, "数据库", "Database Systems", "DATA_STORE_CLASS"), + item(2L, "中文技能", "database technology", "UNRELATED_CODE"), + item(3L, "代码技能", "Code English", "DATABASE_ENGINE") + ); + stubCloud(cloud(items)); + doReturn(List.of(alias(1L, "JVM Language", "jvm language"))) + .when(aliasRepository).findAllBySkillIdInOrderByIdAsc(anyCollection()); + saveProfile(USER_ID, json(List.of( + "jvm-language", + "JVM LANGUAGE", + "数据库", + "DATABASE TECHNOLOGY", + "database_engine" + ))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.matchScore()).isEqualTo(100); + assertThat(result.matchedCount()).isEqualTo(3); + assertThat(result.matchedSkills()).extracting(SkillCloudItem::skillId).containsExactly(1L, 2L, 3L); + assertThat(result.missingSkills()).isEmpty(); + } + + @Test + void matchesOnlyNameZhForV36JavaWhenOtherIdentityFieldsDoNotMatch() { + jdbcTemplate.update("UPDATE skills SET code = ? WHERE id = ?", "JAVA_NAME_ZH_ISOLATED", 1L); + saveProfile(USER_ID, json(List.of("Java"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 2L, 201L); + + assertThat(result.totalCount()).isEqualTo(2); + assertThat(result.matchScore()).isEqualTo(50); + assertThat(result.matchedSkills()) + .extracting(SkillCloudItem::skillId) + .containsExactly(1L); + assertThat(result.missingSkills()) + .extracting(SkillCloudItem::skillId) + .containsExactly(2L); + } + + @Test + void matchesOnlyInsertedAliasForV36JavaWhenAliasIsTheOnlyIdentityMatch() { + jdbcTemplate.update(""" + INSERT INTO skill_aliases (id, skill_id, alias, normalized_alias) + VALUES (?, ?, ?, ?) + """, 10001L, 1L, "ONLY.ALIAS-JVM-XYZ", "only alias jvm xyz"); + saveProfile(USER_ID, json(List.of("ONLY.ALIAS-JVM-XYZ"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 2L, 201L); + + assertThat(result.totalCount()).isEqualTo(2); + assertThat(result.matchScore()).isEqualTo(50); + assertThat(result.matchedSkills()) + .extracting(SkillCloudItem::skillId) + .containsExactly(1L); + assertThat(result.missingSkills()) + .extracting(SkillCloudItem::skillId) + .containsExactly(2L); + } + + @Test + void doesNotApplyAliasOwnedByAnotherTopSkillToV36Java() { + jdbcTemplate.update(""" + INSERT INTO skill_aliases (id, skill_id, alias, normalized_alias) + VALUES (?, ?, ?, ?) + """, 10002L, 2L, "ONLY.ALIAS-OWNER-SKILL-2", "only alias owner skill 2"); + saveProfile(USER_ID, json(List.of("ONLY.ALIAS-OWNER-SKILL-2"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 2L, 201L); + + assertThat(result.totalCount()).isEqualTo(2); + assertThat(result.matchScore()).isEqualTo(50); + assertThat(result.matchedSkills()) + .extracting(SkillCloudItem::skillId) + .containsExactly(2L); + assertThat(result.missingSkills()) + .extracting(SkillCloudItem::skillId) + .containsExactly(1L); + } + + @Test + void keepsMatchedAndMissingListsOrderedAndTheirUnionEqualToTopTen() { + List items = List.of( + item(1L, "Java", "Java", "JAVA"), + item(2L, "Python", "Python", "PYTHON"), + item(3L, "SQL", "SQL", "SQL") + ); + stubCloud(cloud(items)); + saveProfile(USER_ID, json(List.of("Python"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + List union = Stream.concat(result.matchedSkills().stream(), result.missingSkills().stream()) + .map(SkillCloudItem::skillId) + .toList(); + + assertThat(result.matchScore()).isEqualTo(33); + assertThat(result.matchedSkills()).extracting(SkillCloudItem::skillId).containsExactly(2L); + assertThat(result.missingSkills()).extracting(SkillCloudItem::skillId).containsExactly(1L, 3L); + assertThat(union).containsExactlyInAnyOrder(1L, 2L, 3L); + } + + @Test + void returnsZeroWhenNoTopSkillMatches() { + List items = List.of( + item(1L, "Java", "Java", "JAVA"), + item(2L, "SQL", "SQL", "SQL") + ); + stubCloud(cloud(items)); + saveProfile(USER_ID, json(List.of("Marketing"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.matchScore()).isZero(); + assertThat(result.matchedSkills()).isEmpty(); + assertThat(result.missingSkills()).extracting(SkillCloudItem::skillId).containsExactly(1L, 2L); + } + + @Test + void returnsOneHundredAndNoMissingSkillsWhenAllTopSkillsMatch() { + List items = List.of( + item(1L, "Java", "Java", "JAVA"), + item(2L, "SQL", "SQL", "SQL") + ); + stubCloud(cloud(items)); + saveProfile(USER_ID, json(List.of("Java", "SQL"))); + + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + + assertThat(result.matchScore()).isEqualTo(100); + assertThat(result.matchedSkills()).extracting(SkillCloudItem::skillId).containsExactly(1L, 2L); + assertThat(result.missingSkills()).isEmpty(); + } + + @Test + void excludesCrossOccupationCategoryMismatchFromMatchSet() { + jdbcTemplate.update(""" + INSERT INTO occupation_skills ( + id, occupation_id, skill_id, category_id, importance, + importance_source_code, importance_element_id, importance_is_proxy, + relationship_type, source_version_id + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + """, + 10000L, 1L, 1L, 201L, new BigDecimal("5.00"), + "15-1254.00", "2.E.7.h", true, "ESSENTIAL", 2L); + saveProfile(USER_ID, json(List.of("Java"))); + + SkillCloudResponse cloud = queryService.query(1L, 1L, null); + SkillMatchResponse result = service.match(USER_ID, 1L, 1L, null); + List expectedTop = cloud.items().stream().limit(10).map(SkillCloudItem::skillId).toList(); + List actualUnion = Stream.concat(result.matchedSkills().stream(), result.missingSkills().stream()) + .map(SkillCloudItem::skillId) + .toList(); + + assertThat(result.matchedCount()).isZero(); + assertThat(actualUnion).containsExactlyElementsOf(expectedTop); + assertThat(actualUnion).doesNotContain(1L); + } + + private void stubCloud(SkillCloudResponse response) { + doReturn(response).when(queryService).query(any(), any(), any()); + } + + private StudentProfile saveProfile(Long userId, String skillsJson) { + jdbcTemplate.update(""" + INSERT INTO users (id, phone, role, is_active, is_verified, created_at, updated_at) + VALUES (?, ?, 'STUDENT', TRUE, FALSE, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) + """, userId, String.valueOf(userId)); + return profileRepository.saveAndFlush(StudentProfile.builder() + .userId(userId) + .skillsJson(skillsJson) + .build()); + } + + private SkillCloudResponse cloud(List items) { + return new SkillCloudResponse( + new SkillCloudNode(1L, "INTERNET", "互联网"), + new SkillCloudNode(1L, "FRONTEND_DEVELOPER", "前端开发工程师"), + null, + items.size(), + items + ); + } + + private SkillCloudItem item(Long id, String nameZh, String nameEn, String code) { + return new SkillCloudItem( + id.intValue(), + id, + code, + nameZh, + nameEn, + "O*NET", + "test-source", + 1L, + "测试分类", + BigDecimal.TEN, + null, + null, + false + ); + } + + private SkillAlias alias(Long skillId, String alias, String normalizedAlias) { + return SkillAlias.builder() + .id(1L) + .skillId(skillId) + .alias(alias) + .normalizedAlias(normalizedAlias) + .build(); + } + + private String json(List values) { + return values.stream() + .map(value -> "\"" + value.replace("\\", "\\\\").replace("\"", "\\\"") + "\"") + .collect(java.util.stream.Collectors.joining(",", "[", "]")); + } + + public static class SkillMatchSchemaFilterProvider implements SchemaFilterProvider { + + private static final Set VALIDATED_TABLES = Set.of( + "skill_source_versions", + "skill_industries", + "skill_occupations", + "skill_categories", + "skills", + "skill_aliases", + "occupation_skills" + ); + + @Override + public SchemaFilter getCreateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getDropFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getTruncatorFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getMigrateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getValidateFilter() { + return new SchemaFilter() { + @Override + public boolean includeNamespace(Namespace namespace) { + return true; + } + + @Override + public boolean includeTable(Table table) { + return VALIDATED_TABLES.contains(table.getName().toLowerCase(Locale.ROOT)); + } + + @Override + public boolean includeSequence(Sequence sequence) { + return false; + } + }; + } + } +} diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillTaxonomyMigrationTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillTaxonomyMigrationTest.java new file mode 100644 index 0000000..70b4242 --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillTaxonomyMigrationTest.java @@ -0,0 +1,373 @@ +package com.mydreamjob.skill; + +import org.junit.jupiter.api.AfterEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.dao.DataAccessException; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; + +import javax.sql.DataSource; +import java.math.BigDecimal; +import java.sql.Connection; +import java.sql.DatabaseMetaData; +import java.sql.ResultSet; +import java.sql.SQLException; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Locale; +import java.util.Map; +import java.util.TreeMap; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; + +@SpringBootTest +@TestPropertySource(properties = + "spring.datasource.url=jdbc:h2:mem:integrationdb_skill_taxonomy;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE") +@ActiveProfiles("integration-test") +class SkillTaxonomyMigrationTest { + + @Autowired + private JdbcTemplate jdbcTemplate; + + @Autowired + private DataSource dataSource; + + @MockBean + private RedisTemplate redisTemplate; + + private Long sourceVersionId; + private Long industryId; + private Long occupationId; + private Long categoryId; + private Long skillId; + private Long secondSkillId; + private Long occupationSkillId; + + @Test + void migrationCreatesTaxonomyTablesColumnsConstraintsAndIndexes() throws SQLException { + assertTableAndColumns("skill_source_versions", + "id", "source_name", "source_version", "license", "attribution_url", + "imported_at", "created_at", "updated_at"); + assertTableAndColumns("skill_industries", + "id", "code", "name_zh", "source_code", "sort_order", "created_at", "updated_at"); + assertTableAndColumns("skill_occupations", + "id", "industry_id", "code", "name_zh", "name_en", "source_code", + "source_version_id", "sort_order", "created_at", "updated_at"); + assertTableAndColumns("skill_categories", + "id", "occupation_id", "code", "name_zh", "suggestion_template_code", + "sort_order", "created_at", "updated_at"); + assertTableAndColumns("skills", + "id", "code", "name_zh", "name_en", "source_type", "source_code", + "source_version_id", "sort_order", "created_at", "updated_at"); + assertTableAndColumns("skill_aliases", + "id", "skill_id", "alias", "normalized_alias", "created_at"); + assertTableAndColumns("occupation_skills", + "id", "occupation_id", "skill_id", "category_id", "importance", "importance_source_code", "importance_element_id", "importance_is_proxy", + "relationship_type", "source_version_id", "created_at", "updated_at"); + + assertUniqueIndex("skill_source_versions", "source_name", "source_version"); + assertUniqueIndex("skill_industries", "code"); + assertUniqueIndex("skill_occupations", "industry_id", "code"); + assertUniqueIndex("skill_categories", "occupation_id", "code"); + assertUniqueIndex("skills", "code"); + assertUniqueIndex("skills", "source_type", "source_code"); + assertUniqueIndex("skill_aliases", "normalized_alias"); + assertUniqueIndex("skill_aliases", "skill_id", "alias"); + assertUniqueIndex("occupation_skills", "occupation_id", "skill_id"); + + assertForeignKey("skill_occupations", "industry_id", "skill_industries", "id"); + assertForeignKey("skill_occupations", "source_version_id", "skill_source_versions", "id"); + assertForeignKey("skill_categories", "occupation_id", "skill_occupations", "id"); + assertForeignKey("skills", "source_version_id", "skill_source_versions", "id"); + assertForeignKey("skill_aliases", "skill_id", "skills", "id"); + assertForeignKey("occupation_skills", "occupation_id", "skill_occupations", "id"); + assertForeignKey("occupation_skills", "skill_id", "skills", "id"); + assertForeignKey("occupation_skills", "category_id", "skill_categories", "id"); + assertForeignKey("occupation_skills", "source_version_id", "skill_source_versions", "id"); + + assertIndex("occupation_skills", "occupation_id", "category_id"); + assertIndex("occupation_skills", "occupation_id", "importance"); + assertBooleanColumnMetadata("occupation_skills", "importance_is_proxy"); + assertEquals(1L, jdbcTemplate.queryForObject( + "SELECT COUNT(*) FROM information_schema.table_constraints " + + "WHERE UPPER(table_name) = 'OCCUPATION_SKILLS' " + + "AND UPPER(constraint_name) = 'CK_OCCUPATION_SKILLS_IMPORTANCE_PROXY' " + + "AND UPPER(constraint_type) = 'CHECK'", + Long.class)); + } + + @Test + void minimumTaxonomyChainIsUsableAndCoversCoreConstraints() { + sourceVersionId = insertAndReturnId( + "INSERT INTO skill_source_versions " + + "(source_name, source_version, license, attribution_url, imported_at) " + + "VALUES (?, ?, ?, ?, CURRENT_TIMESTAMP)", + "ESCO", "test-1.2.1", "Commission Decision 2011/833/EU", + "https://esco.ec.europa.eu/en/use-esco/download"); + industryId = insertAndReturnId( + "INSERT INTO skill_industries (code, name_zh, source_code, sort_order) " + + "VALUES (?, ?, ?, ?)", + "TEST_INTERNET", "互联网", "PROJECT-INTERNET", 1); + occupationId = insertAndReturnId( + "INSERT INTO skill_occupations " + + "(industry_id, code, name_zh, name_en, source_code, source_version_id, sort_order) " + + "VALUES (?, ?, ?, ?, ?, ?, ?)", + industryId, "TEST_FRONTEND", "前端开发工程师", "Frontend Developer", + "ESCO-2513.2", sourceVersionId, 1); + categoryId = insertAndReturnId( + "INSERT INTO skill_categories " + + "(occupation_id, code, name_zh, suggestion_template_code, sort_order) " + + "VALUES (?, ?, ?, ?, ?)", + occupationId, "TEST_PROGRAMMING_LANGUAGE", "编程语言", + "PROGRAMMING_LANGUAGE", 1); + skillId = insertAndReturnId( + "INSERT INTO skills " + + "(code, name_zh, name_en, source_type, source_code, source_version_id, sort_order) " + + "VALUES (?, ?, ?, ?, ?, ?, ?)", + "TEST_SKILL_JAVA", "Java", "Java", "ESCO", "ESCO-SKILL-JAVA", + sourceVersionId, 1); + insertAndReturnId( + "INSERT INTO skill_aliases (skill_id, alias, normalized_alias) VALUES (?, ?, ?)", + skillId, "Java SE", "java se"); + occupationSkillId = insertAndReturnId( + "INSERT INTO occupation_skills " + + "(occupation_id, skill_id, category_id, importance, relationship_type, source_version_id) " + + "VALUES (?, ?, ?, ?, ?, ?)", + occupationId, skillId, categoryId, null, "ESSENTIAL", sourceVersionId); + + Map joined = jdbcTemplate.queryForMap( + "SELECT s.name_zh AS skill_name, c.name_zh AS category_name, os.importance, " + + "os.importance_source_code, os.importance_element_id, os.importance_is_proxy " + + "FROM occupation_skills os " + + "JOIN skills s ON s.id = os.skill_id " + + "JOIN skill_categories c ON c.id = os.category_id " + + "WHERE os.id = ?", occupationSkillId); + assertEquals("Java", joined.get("skill_name")); + assertEquals("编程语言", joined.get("category_name")); + assertNull(joined.get("importance")); + assertNull(joined.get("importance_source_code")); + assertNull(joined.get("importance_element_id")); + assertEquals(Boolean.FALSE, joined.get("importance_is_proxy")); + assertEquals(1L, jdbcTemplate.queryForObject( + "SELECT COUNT(*) FROM occupation_skills WHERE id = ? AND importance_is_proxy = FALSE", + Long.class, occupationSkillId)); + + assertConstraintFailure("UPDATE occupation_skills SET importance = ? WHERE id = ?", + new BigDecimal("4.25"), occupationSkillId); + jdbcTemplate.update("UPDATE occupation_skills SET importance = ?, " + + "importance_source_code = ?, importance_element_id = ?, importance_is_proxy = ? " + + "WHERE id = ?", + new BigDecimal("4.25"), "15-1252.00", "4.A.3.b.1", Boolean.TRUE, occupationSkillId); + Map weighted = jdbcTemplate.queryForMap( + "SELECT importance, importance_source_code, importance_element_id, importance_is_proxy " + + "FROM occupation_skills WHERE id = ?", + occupationSkillId); + assertEquals(new BigDecimal("4.25"), weighted.get("importance")); + assertEquals("15-1252.00", weighted.get("importance_source_code")); + assertEquals("4.A.3.b.1", weighted.get("importance_element_id")); + assertEquals(Boolean.TRUE, weighted.get("importance_is_proxy")); + assertEquals(1L, jdbcTemplate.queryForObject( + "SELECT COUNT(*) FROM occupation_skills WHERE id = ? AND importance_is_proxy = TRUE", + Long.class, occupationSkillId)); + assertConstraintFailure( + "INSERT INTO skill_source_versions " + + "(source_name, source_version, license, attribution_url) VALUES (?, ?, ?, ?)", + "ESCO", "test-1.2.1", "duplicate", "https://example.invalid"); + assertConstraintFailure( + "INSERT INTO skill_categories " + + "(occupation_id, code, name_zh, suggestion_template_code, sort_order) " + + "VALUES (?, ?, ?, ?, ?)", + occupationId, "TEST_INVALID_CATEGORY", "非法分类", "INVALID_TEMPLATE", 2); + assertConstraintFailure( + "INSERT INTO skills " + + "(code, name_zh, source_type, source_code, source_version_id, sort_order) " + + "VALUES (?, ?, ?, ?, ?, ?)", + "TEST_INVALID_SKILL", "非法技能", "INVALID_SOURCE", "INVALID-SOURCE-1", + sourceVersionId, 2); + secondSkillId = insertAndReturnId( + "INSERT INTO skills " + + "(code, name_zh, source_type, source_code, source_version_id, sort_order) " + + "VALUES (?, ?, ?, ?, ?, ?)", + "TEST_SKILL_SPRING", "Spring Boot", "ESCO", "ESCO-SKILL-SPRING", + sourceVersionId, 2); + assertConstraintFailure( + "INSERT INTO skills " + + "(code, name_zh, source_type, source_code, source_version_id, sort_order) " + + "VALUES (?, ?, ?, ?, ?, ?)", + "TEST_SKILL_NULL_NAME", null, "PROJECT", "PROJECT-NULL-NAME", + sourceVersionId, 3); + assertConstraintFailure( + "INSERT INTO occupation_skills " + + "(occupation_id, skill_id, category_id, relationship_type, source_version_id) " + + "VALUES (?, ?, ?, ?, ?)", + occupationId, secondSkillId, categoryId, "INVALID_RELATIONSHIP", sourceVersionId); + assertConstraintFailure( + "INSERT INTO skill_aliases (skill_id, alias, normalized_alias) VALUES (?, ?, ?)", + secondSkillId, "Java SE alternate", "java se"); + + assertConstraintFailure( + "UPDATE occupation_skills SET importance_is_proxy = ? WHERE id = ?", + (Object) null, occupationSkillId); + assertTrue(jdbcTemplate.queryForObject( + "SELECT COUNT(*) FROM skill_aliases WHERE skill_id = ? AND normalized_alias = ?", + Integer.class, skillId, "java se") == 1); + } + + @AfterEach + void cleanUp() { + if (occupationSkillId != null) { + jdbcTemplate.update("DELETE FROM occupation_skills WHERE id = ?", occupationSkillId); + } + if (secondSkillId != null) { + jdbcTemplate.update("DELETE FROM skill_aliases WHERE skill_id = ?", secondSkillId); + jdbcTemplate.update("DELETE FROM skills WHERE id = ?", secondSkillId); + } + if (skillId != null) { + jdbcTemplate.update("DELETE FROM skill_aliases WHERE skill_id = ?", skillId); + jdbcTemplate.update("DELETE FROM skills WHERE id = ?", skillId); + } + if (categoryId != null) { + jdbcTemplate.update("DELETE FROM skill_categories WHERE id = ?", categoryId); + } + if (occupationId != null) { + jdbcTemplate.update("DELETE FROM skill_occupations WHERE id = ?", occupationId); + } + if (industryId != null) { + jdbcTemplate.update("DELETE FROM skill_industries WHERE id = ?", industryId); + } + if (sourceVersionId != null) { + jdbcTemplate.update("DELETE FROM skill_source_versions WHERE id = ?", sourceVersionId); + } + } + + private void assertBooleanColumnMetadata(String table, String column) { + Map metadata = jdbcTemplate.queryForMap( + "SELECT data_type, column_default, is_nullable FROM information_schema.columns " + + "WHERE UPPER(table_name) = ? AND UPPER(column_name) = ?", + table.toUpperCase(Locale.ROOT), column.toUpperCase(Locale.ROOT)); + assertEquals("BOOLEAN", String.valueOf(metadata.get("data_type")).toUpperCase(Locale.ROOT)); + assertEquals("FALSE", String.valueOf(metadata.get("column_default")).toUpperCase(Locale.ROOT)); + assertEquals("NO", String.valueOf(metadata.get("is_nullable")).toUpperCase(Locale.ROOT)); + } + + private void assertTableAndColumns(String table, String... columns) throws SQLException { + try (Connection connection = dataSource.getConnection()) { + DatabaseMetaData metadata = connection.getMetaData(); + String actualTable = findTable(metadata, table); + assertTrue(actualTable != null, "Missing table: " + table); + + List actualColumns = new ArrayList<>(); + try (ResultSet resultSet = metadata.getColumns(null, null, actualTable, null)) { + while (resultSet.next()) { + actualColumns.add(resultSet.getString("COLUMN_NAME").toLowerCase()); + } + } + for (String column : columns) { + assertTrue(actualColumns.contains(column.toLowerCase()), + "Missing column " + table + "." + column + "; actual=" + actualColumns); + } + } + } + + private void assertUniqueIndex(String table, String... columns) throws SQLException { + assertTrue(hasIndex(table, true, columns), + "Missing unique index on " + table + "(" + String.join(", ", columns) + ")"); + } + + private void assertIndex(String table, String... columns) throws SQLException { + assertTrue(hasIndex(table, false, columns), + "Missing index on " + table + "(" + String.join(", ", columns) + ")"); + } + + private boolean hasIndex(String table, boolean unique, String... expectedColumns) throws SQLException { + try (Connection connection = dataSource.getConnection()) { + DatabaseMetaData metadata = connection.getMetaData(); + String actualTable = findTable(metadata, table); + if (actualTable == null) { + return false; + } + Map> indexes = new LinkedHashMap<>(); + try (ResultSet resultSet = metadata.getIndexInfo(null, null, actualTable, false, false)) { + while (resultSet.next()) { + String indexName = resultSet.getString("INDEX_NAME"); + if (indexName == null || resultSet.getBoolean("NON_UNIQUE") == unique) { + continue; + } + indexes.computeIfAbsent(indexName, ignored -> new TreeMap<>()) + .put(resultSet.getShort("ORDINAL_POSITION"), + resultSet.getString("COLUMN_NAME").toLowerCase()); + } + } + List expected = new ArrayList<>(); + for (String column : expectedColumns) { + expected.add(column.toLowerCase()); + } + return indexes.values().stream() + .map(TreeMap::values) + .map(values -> new ArrayList<>(values)) + .anyMatch(expected::equals); + } + } + + private void assertForeignKey(String childTable, String childColumn, + String parentTable, String parentColumn) throws SQLException { + try (Connection connection = dataSource.getConnection()) { + DatabaseMetaData metadata = connection.getMetaData(); + String actualChildTable = findTable(metadata, childTable); + String actualParentTable = findTable(metadata, parentTable); + if (actualChildTable == null || actualParentTable == null) { + throw new AssertionError("Missing table for foreign key " + childTable + "." + childColumn); + } + try (ResultSet resultSet = metadata.getImportedKeys(null, null, actualChildTable)) { + while (resultSet.next()) { + if (actualParentTable.equalsIgnoreCase(resultSet.getString("PKTABLE_NAME")) + && childColumn.equalsIgnoreCase(resultSet.getString("FKCOLUMN_NAME")) + && parentColumn.equalsIgnoreCase(resultSet.getString("PKCOLUMN_NAME"))) { + return; + } + } + } + } + throw new AssertionError("Missing foreign key " + childTable + "." + childColumn + + " -> " + parentTable + "." + parentColumn); + } + + private String findTable(DatabaseMetaData metadata, String expected) throws SQLException { + try (ResultSet resultSet = metadata.getTables(null, null, null, new String[]{"TABLE"})) { + while (resultSet.next()) { + String actual = resultSet.getString("TABLE_NAME"); + if (expected.equalsIgnoreCase(actual)) { + return actual; + } + } + } + return null; + } + + private Long insertAndReturnId(String sql, Object... args) { + jdbcTemplate.update(sql, args); + String table = extractTable(sql); + return jdbcTemplate.queryForObject("SELECT MAX(id) FROM " + table, Long.class); + } + + private String extractTable(String sql) { + String upper = sql.toUpperCase(); + int fromIndex = upper.indexOf(" INTO "); + int openIndex = sql.indexOf('(', fromIndex); + return sql.substring(fromIndex + " INTO ".length(), openIndex).trim(); + } + + private void assertConstraintFailure(String sql, Object... args) { + assertThrows(DataAccessException.class, () -> jdbcTemplate.update(sql, args)); + } +} + diff --git a/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillTaxonomyRepositoryTest.java b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillTaxonomyRepositoryTest.java new file mode 100644 index 0000000..40f4103 --- /dev/null +++ b/mydreamjob-server/src/test/java/com/mydreamjob/skill/SkillTaxonomyRepositoryTest.java @@ -0,0 +1,294 @@ +package com.mydreamjob.skill; + +import com.mydreamjob.skill.domain.OccupationSkill; +import com.mydreamjob.skill.domain.Skill; +import com.mydreamjob.skill.domain.SkillAlias; +import com.mydreamjob.skill.dto.SkillCloudSkillProjection; +import com.mydreamjob.skill.repository.OccupationSkillRepository; +import com.mydreamjob.skill.repository.SkillAliasRepository; +import com.mydreamjob.skill.repository.SkillCategoryRepository; +import com.mydreamjob.skill.repository.SkillIndustryRepository; +import com.mydreamjob.skill.repository.SkillOccupationRepository; +import com.mydreamjob.skill.repository.SkillRepository; +import com.mydreamjob.skill.repository.SkillSourceVersionRepository; +import jakarta.persistence.EntityManagerFactory; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.mock.mockito.MockBean; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.jdbc.core.JdbcTemplate; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; +import org.springframework.transaction.annotation.Transactional; +import org.hibernate.engine.spi.SessionFactoryImplementor; +import org.hibernate.boot.model.relational.Namespace; +import org.hibernate.boot.model.relational.Sequence; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.persister.entity.EntityPersister; +import org.hibernate.mapping.Table; +import org.hibernate.tool.schema.spi.SchemaFilter; +import org.hibernate.tool.schema.spi.SchemaFilterProvider; + +import java.math.BigDecimal; +import java.util.List; +import java.util.Locale; +import java.util.Optional; +import java.util.Set; + +import static org.assertj.core.api.Assertions.assertThat; + +@SpringBootTest +@TestPropertySource(properties = { + "spring.datasource.url=jdbc:h2:mem:integrationdb_skill_repository;MODE=MySQL;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE", + "spring.jpa.hibernate.ddl-auto=validate", + "spring.jpa.properties.hibernate.hbm2ddl.schema_filter_provider=com.mydreamjob.skill.SkillTaxonomyRepositoryTest$SkillTaxonomySchemaFilterProvider" +}) +@ActiveProfiles("integration-test") +class SkillTaxonomyRepositoryTest { + + @Autowired + private SkillSourceVersionRepository sourceVersionRepository; + + @Autowired + private SkillIndustryRepository industryRepository; + + @Autowired + private SkillOccupationRepository occupationRepository; + + @Autowired + private SkillCategoryRepository categoryRepository; + + @Autowired + private SkillRepository skillRepository; + + @Autowired + private SkillAliasRepository skillAliasRepository; + + @Autowired + private OccupationSkillRepository occupationSkillRepository; + + @Autowired + private JdbcTemplate jdbcTemplate; + + @Autowired + private EntityManagerFactory entityManagerFactory; + + @MockBean + private RedisTemplate redisTemplate; + + @Test + void entityMappingsAreValidatedAndSeedCountsMatchV36() { + assertThat(sourceVersionRepository.count()).isEqualTo(2L); + assertThat(industryRepository.count()).isEqualTo(3L); + assertThat(occupationRepository.count()).isEqualTo(10L); + assertThat(categoryRepository.count()).isEqualTo(68L); + assertThat(skillRepository.count()).isEqualTo(72L); + assertThat(skillAliasRepository.count()).isEqualTo(72L); + assertThat(occupationSkillRepository.count()).isEqualTo(139L); + } + + @Test + void associationChecksAcceptOnlyConsistentTaxonomyPaths() { + assertThat(industryRepository.existsById(1L)).isTrue(); + assertThat(occupationRepository.existsByIdAndIndustryId(1L, 1L)).isTrue(); + assertThat(categoryRepository.existsByIdAndOccupationId(101L, 1L)).isTrue(); + + assertThat(occupationRepository.existsByIdAndIndustryId(1L, 2L)).isFalse(); + assertThat(categoryRepository.existsByIdAndOccupationId(101L, 2L)).isFalse(); + assertThat(occupationRepository.existsByIdAndIndustryId(999L, 1L)).isFalse(); + assertThat(categoryRepository.existsByIdAndOccupationId(999L, 1L)).isFalse(); + } + + @Test + void projectionUsesWholeOccupationWhenCategoryMissingAndFiltersWhenProvided() { + List occupationSkills = + occupationSkillRepository.findSkillProjectionsByOccupationId(1L); + + assertThat(occupationSkills).hasSize(15); + assertThat(occupationSkills) + .extracting(SkillCloudSkillProjection::skillId) + .containsExactlyInAnyOrder( + 3L, 4L, 7L, 8L, 26L, 27L, 28L, 35L, + 54L, 55L, 38L, 39L, 32L, 24L, 50L); + + List languageSkills = + occupationSkillRepository.findSkillProjectionsByOccupationIdAndCategoryId(1L, 101L); + + assertThat(languageSkills).hasSize(2); + assertThat(languageSkills) + .extracting(SkillCloudSkillProjection::skillId) + .containsExactlyInAnyOrder(3L, 4L); + assertThat(languageSkills) + .extracting(SkillCloudSkillProjection::categoryId) + .containsOnly(101L); + + assertThat(occupationSkillRepository + .findSkillProjectionsByOccupationIdAndCategoryId(2L, 101L)) + .isEmpty(); + } + + @Test + @Transactional + void projectionQueriesRejectCrossOccupationCategoryMismatch() { + jdbcTemplate.update(""" + INSERT INTO occupation_skills ( + occupation_id, skill_id, category_id, importance, + importance_source_code, importance_element_id, + importance_is_proxy, relationship_type, source_version_id + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) + """, + 1L, 1L, 201L, new BigDecimal("4.61"), + "15-1252.00", "4.A.3.b.1", Boolean.TRUE, "ESSENTIAL", 2L); + + List occupationOneSkills = + occupationSkillRepository.findSkillProjectionsByOccupationId(1L); + + assertThat(occupationOneSkills) + .extracting(SkillCloudSkillProjection::skillId) + .contains(3L) + .doesNotContain(1L); + + assertThat(occupationSkillRepository + .findSkillProjectionsByOccupationIdAndCategoryId(1L, 201L)) + .isEmpty(); + + assertThat(occupationSkillRepository + .findSkillProjectionsByOccupationIdAndCategoryId(2L, 201L)) + .extracting(SkillCloudSkillProjection::skillId) + .contains(1L); + } + + @Test + void projectionCarriesOnetProvenanceNullImportanceAndSortOrders() { + SkillCloudSkillProjection react = occupationSkillRepository + .findSkillProjectionsByOccupationIdAndCategoryId(1L, 103L) + .stream() + .filter(item -> item.skillId().equals(26L)) + .findFirst() + .orElseThrow(); + + assertThat(react.skillCode()).isEqualTo("REACT"); + assertThat(react.nameZh()).isEqualTo("React"); + assertThat(react.nameEn()).isEqualTo("React"); + assertThat(react.sourceType()).isEqualTo("O*NET"); + assertThat(react.sourceCode()).isEqualTo("15-1254.00|React|2.E.7.h"); + assertThat(react.categoryName()).isEqualTo("前端框架"); + assertThat(react.importance()).isEqualByComparingTo(new BigDecimal("4.88")); + assertThat(react.importanceSourceCode()).isEqualTo("15-1254.00"); + assertThat(react.importanceElementId()).isEqualTo("4.A.3.b.1"); + assertThat(react.importanceIsProxy()).isTrue(); + assertThat(react.skillSortOrder()).isEqualTo(26); + assertThat(react.categorySortOrder()).isEqualTo(3); + + SkillCloudSkillProjection figma = occupationSkillRepository + .findSkillProjectionsByOccupationIdAndCategoryId(1L, 104L) + .stream() + .filter(item -> item.skillId().equals(50L)) + .findFirst() + .orElseThrow(); + + assertThat(figma.sourceType()).isEqualTo("O*NET"); + assertThat(figma.sourceCode()).isEqualTo("15-1254.00|Figma|2.E.7.d"); + assertThat(figma.importance()).isNull(); + assertThat(figma.importanceSourceCode()).isNull(); + assertThat(figma.importanceElementId()).isNull(); + assertThat(figma.importanceIsProxy()).isFalse(); + } + + @Test + void aliasAndSkillLookupsReturnOnlyRequestedRecords() { + Optional normalizedAlias = skillAliasRepository.findByNormalizedAlias("react"); + assertThat(normalizedAlias).isPresent(); + assertThat(normalizedAlias.orElseThrow().getSkillId()).isEqualTo(26L); + + List aliases = skillAliasRepository + .findAllBySkillIdInOrderByIdAsc(List.of(3L, 26L)); + assertThat(aliases).hasSize(2); + assertThat(aliases) + .extracting(SkillAlias::getSkillId) + .containsExactly(3L, 26L); + + Optional react = skillRepository.findByCode("REACT"); + assertThat(react).isPresent(); + assertThat(react.orElseThrow().getSourceType()).isEqualTo("O*NET"); + + List skills = skillRepository.findAllByIdInOrderByIdAsc(List.of(4L, 3L)); + assertThat(skills) + .extracting(Skill::getId) + .containsExactly(3L, 4L); + } + + @Test + void importanceIsProxyUsesDefaultBooleanMapping() throws NoSuchFieldException { + SessionFactoryImplementor sessionFactory = entityManagerFactory.unwrap(SessionFactoryImplementor.class); + EntityPersister persister = sessionFactory.getMappingMetamodel().getEntityDescriptor(OccupationSkill.class); + + org.hibernate.type.Type propertyType = persister.getPropertyType("importanceIsProxy"); + assertThat(propertyType.getReturnedClass()).isEqualTo(Boolean.class); + + JdbcTypeCode explicitJdbcType = OccupationSkill.class + .getDeclaredField("importanceIsProxy") + .getAnnotation(JdbcTypeCode.class); + assertThat(explicitJdbcType) + .as("importanceIsProxy must use the default Boolean mapping without a forced JDBC type") + .isNull(); + } + + /** + * The legacy H2 schema has TINYINT/Boolean mismatches outside the V35 taxonomy. + * Scope validation to the seven T4 tables without changing historical migrations or entities. + */ + public static class SkillTaxonomySchemaFilterProvider implements SchemaFilterProvider { + + private static final Set VALIDATED_TABLES = Set.of( + "skill_source_versions", + "skill_industries", + "skill_occupations", + "skill_categories", + "skills", + "skill_aliases", + "occupation_skills" + ); + + @Override + public SchemaFilter getCreateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getDropFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getTruncatorFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getMigrateFilter() { + return SchemaFilter.ALL; + } + + @Override + public SchemaFilter getValidateFilter() { + return new SchemaFilter() { + @Override + public boolean includeNamespace(Namespace namespace) { + return true; + } + + @Override + public boolean includeTable(Table table) { + return VALIDATED_TABLES.contains(table.getName().toLowerCase(Locale.ROOT)); + } + + @Override + public boolean includeSequence(Sequence sequence) { + return false; + } + }; + } + } +} diff --git a/mydreamjob-server/src/test/resources/skill-cloud/onet-31-reference.csv b/mydreamjob-server/src/test/resources/skill-cloud/onet-31-reference.csv new file mode 100644 index 0000000..2a4e1b2 --- /dev/null +++ b/mydreamjob-server/src/test/resources/skill-cloud/onet-31-reference.csv @@ -0,0 +1,93 @@ +evidence_type,source_soc,element_id,data_value,workplace_example,source_file +SOFTWARE,11-1011.00,2.E.5.b,,Adobe Acrobat,software_skills.csv +SOFTWARE,11-1021.00,2.E.6.f,,Amazon Web Services AWS software,software_skills.csv +SOFTWARE,11-3021.00,2.E.11.a,,Nagios,software_skills.csv +SOFTWARE,13-1199.07,2.E.14.d,,Nmap,software_skills.csv +SOFTWARE,15-1241.01,2.E.11.a,,Wireshark,software_skills.csv +SOFTWARE,15-1252.00,2.E.13.c,,Bash,software_skills.csv +SOFTWARE,15-1252.00,2.E.7.e,,C++,software_skills.csv +SOFTWARE,15-1252.00,2.E.7.h,,Django,software_skills.csv +SOFTWARE,15-1252.00,2.E.10.a,,Docker,software_skills.csv +SOFTWARE,15-1252.00,2.E.5.c,,Git,software_skills.csv +SOFTWARE,15-1252.00,2.E.7.b,,Go,software_skills.csv +SOFTWARE,15-1252.00,2.E.7.e,,Gradle,software_skills.csv +SOFTWARE,15-1252.00,2.E.10.a,,Kubernetes,software_skills.csv +SOFTWARE,15-1252.00,2.E.6.d,,MongoDB,software_skills.csv +SOFTWARE,15-1252.00,2.E.6.d,,Redis,software_skills.csv +SOFTWARE,15-1252.00,2.E.7.f,,SonarQube,software_skills.csv +SOFTWARE,15-1252.00,2.E.10.a,,Spring Boot,software_skills.csv +SOFTWARE,15-1253.00,2.E.7.e,,Apache JMeter,software_skills.csv +SOFTWARE,15-1253.00,2.E.7.f,,JUnit,software_skills.csv +SOFTWARE,15-1253.00,2.E.13.c,,Postman,software_skills.csv +SOFTWARE,15-1253.00,2.E.7.f,,Selenium,software_skills.csv +SOFTWARE,15-1254.00,2.E.7.d,,Figma,software_skills.csv +SOFTWARE,15-1254.00,2.E.7.h,,Node.js,software_skills.csv +SOFTWARE,15-1254.00,2.E.7.h,,React,software_skills.csv +SOFTWARE,15-1254.00,2.E.7.h,,Vue.js,software_skills.csv +SOFTWARE,15-1255.00,2.E.4.g,,Axure RP,software_skills.csv +SOFTWARE,15-1255.00,2.E.7.a,,Chef,software_skills.csv +SOFTWARE,15-1299.07,2.E.7.b,,Apache Kafka,software_skills.csv +SOFTWARE,15-1299.08,2.E.10.a,,GitLab,software_skills.csv +SOFTWARE,15-2051.00,2.E.6.m,,Apache Spark,software_skills.csv +SOFTWARE,15-2051.00,2.E.7.b,,Flask,software_skills.csv +SOFTWARE,15-2051.00,2.E.4.i,,Microsoft Excel,software_skills.csv +SOFTWARE,15-2051.00,2.E.6.m,,Microsoft Power BI,software_skills.csv +SOFTWARE,15-2051.00,2.E.6.f,,NumPy,software_skills.csv +SOFTWARE,15-2051.00,2.E.6.f,,PyTorch,software_skills.csv +SOFTWARE,15-2051.00,2.E.7.b,,Scikit-learn,software_skills.csv +SOFTWARE,15-2051.00,2.E.6.m,,Tableau,software_skills.csv +SOFTWARE,15-2051.00,2.E.9.d,,TensorFlow,software_skills.csv +SOFTWARE,15-2051.00,2.E.6.f,,pandas,software_skills.csv +SOFTWARE,17-2051.02,2.E.10.e,,Google Chrome,software_skills.csv +SOFTWARE,17-2071.00,2.E.7.f,,Debugging software,software_skills.csv +SOFTWARE,19-3092.00,2.E.4.b,,Adobe Photoshop,software_skills.csv +SOFTWARE,25-2022.00,2.E.8.b,,Moodle,software_skills.csv +SOFTWARE,25-9031.00,2.E.8.b,,Articulate Storyline,software_skills.csv +SOFTWARE,27-4012.00,2.E.4.f,,Microsoft PowerPoint,software_skills.csv +SOFTWARE,41-3031.00,2.E.2.d,,Bloomberg Professional,software_skills.csv +SOFTWARE,41-4011.00,2.E.7.b,,Microsoft Azure software,software_skills.csv +IMPORTANCE,13-1082.00,2.C.6,2.52,,knowledge.csv +IMPORTANCE,13-1082.00,4.A.2.a.4,3.94,,work_activities.csv +IMPORTANCE,13-1082.00,4.A.2.b.4,3.73,,work_activities.csv +IMPORTANCE,13-1082.00,4.A.3.b.1,4.45,,work_activities.csv +IMPORTANCE,13-1082.00,4.A.4.a.2,4.73,,work_activities.csv +IMPORTANCE,13-2031.00,2.C.1.c,4.42,,knowledge.csv +IMPORTANCE,13-2031.00,2.C.4.a,4.03,,knowledge.csv +IMPORTANCE,13-2031.00,4.A.2.a.2,4.44,,work_activities.csv +IMPORTANCE,13-2031.00,4.A.2.a.4,4.01,,work_activities.csv +IMPORTANCE,13-2031.00,4.A.3.b.1,4.48,,work_activities.csv +IMPORTANCE,13-2053.00,2.C.4.a,3.27,,knowledge.csv +IMPORTANCE,13-2053.00,4.A.2.a.2,4.13,,work_activities.csv +IMPORTANCE,13-2053.00,4.A.2.a.3,3.90,,work_activities.csv +IMPORTANCE,13-2053.00,4.A.2.a.4,4.43,,work_activities.csv +IMPORTANCE,13-2053.00,4.A.3.b.1,4.41,,work_activities.csv +IMPORTANCE,13-2099.01,2.C.1.c,4.20,,knowledge.csv +IMPORTANCE,13-2099.01,4.A.1.b.3,3.53,,work_activities.csv +IMPORTANCE,13-2099.01,4.A.2.a.2,4.50,,work_activities.csv +IMPORTANCE,13-2099.01,4.A.2.a.3,3.05,,work_activities.csv +IMPORTANCE,13-2099.01,4.A.3.b.1,4.68,,work_activities.csv +IMPORTANCE,15-1244.00,2.C.8.a,2.34,,knowledge.csv +IMPORTANCE,15-1244.00,4.A.1.a.2,3.60,,work_activities.csv +IMPORTANCE,15-1244.00,4.A.2.a.2,4.03,,work_activities.csv +IMPORTANCE,15-1244.00,4.A.3.b.1,4.83,,work_activities.csv +IMPORTANCE,15-1252.00,2.C.8.a,2.57,,knowledge.csv +IMPORTANCE,15-1252.00,4.A.2.a.2,4.38,,work_activities.csv +IMPORTANCE,15-1252.00,4.A.2.a.3,3.94,,work_activities.csv +IMPORTANCE,15-1252.00,4.A.3.b.1,4.61,,work_activities.csv +IMPORTANCE,15-1253.00,4.A.1.b.3,2.94,,work_activities.csv +IMPORTANCE,15-1253.00,4.A.2.a.2,4.38,,work_activities.csv +IMPORTANCE,15-1253.00,4.A.2.a.3,4.24,,work_activities.csv +IMPORTANCE,15-1253.00,4.A.3.b.1,4.94,,work_activities.csv +IMPORTANCE,15-1254.00,4.A.2.a.3,3.33,,work_activities.csv +IMPORTANCE,15-1254.00,4.A.3.b.1,4.88,,work_activities.csv +IMPORTANCE,15-1254.00,4.A.4.a.2,3.88,,work_activities.csv +IMPORTANCE,15-2051.00,2.C.4.a,4.52,,knowledge.csv +IMPORTANCE,15-2051.00,2.C.6,2.28,,knowledge.csv +IMPORTANCE,15-2051.00,4.A.2.a.2,4.71,,work_activities.csv +IMPORTANCE,15-2051.00,4.A.2.a.4,4.83,,work_activities.csv +IMPORTANCE,15-2051.00,4.A.3.b.1,4.75,,work_activities.csv +IMPORTANCE,25-9031.00,2.C.6,4.78,,knowledge.csv +IMPORTANCE,25-9031.00,4.A.2.a.3,3.83,,work_activities.csv +IMPORTANCE,25-9031.00,4.A.2.a.4,4.19,,work_activities.csv +IMPORTANCE,25-9031.00,4.A.2.b.2,3.72,,work_activities.csv +IMPORTANCE,25-9031.00,4.A.3.b.1,4.45,,work_activities.csv diff --git a/mydreamjob-server/src/test/resources/skill-cloud/skill-cloud-expected-mappings.csv b/mydreamjob-server/src/test/resources/skill-cloud/skill-cloud-expected-mappings.csv new file mode 100644 index 0000000..81238cc --- /dev/null +++ b/mydreamjob-server/src/test/resources/skill-cloud/skill-cloud-expected-mappings.csv @@ -0,0 +1,222 @@ +record_type,entity_key,expected_source_type,expected_source_code,expected_element_id,expected_importance,expected_is_proxy +OCCUPATION_SOURCE,FRONTEND_DEVELOPER,ESCO,http://data.europa.eu/esco/occupation/c40a2919-48a9-40ea-b506-1f34f693496d,,, +OCCUPATION_SOURCE,BACKEND_DEVELOPER,ESCO,http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1,,, +OCCUPATION_SOURCE,QA_ENGINEER,ESCO,http://data.europa.eu/esco/occupation/106f79e4-6264-45f1-9e7a-297435cd684b,,, +OCCUPATION_SOURCE,DEVOPS_ENGINEER,ESCO,http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1,,, +OCCUPATION_SOURCE,FINANCIAL_DATA_ANALYST,ESCO,http://data.europa.eu/esco/occupation/8d586ee9-0ab1-4155-a3b0-2ca786c8e48c,,, +OCCUPATION_SOURCE,RISK_MODELING_ENGINEER,ESCO,http://data.europa.eu/esco/occupation/b09acdda-6680-49d1-a411-129610ee3bc0,,, +OCCUPATION_SOURCE,QUANTITATIVE_DEVELOPER,ESCO,http://data.europa.eu/esco/occupation/f2b15a0e-e65a-438a-affb-29b9d50b77d1,,, +OCCUPATION_SOURCE,EDUCATION_PRODUCT_MANAGER,ESCO,http://data.europa.eu/esco/occupation/a79170ad-0b7b-4697-864f-1748359aa10a,,, +OCCUPATION_SOURCE,INSTRUCTIONAL_DESIGNER,ESCO,http://data.europa.eu/esco/occupation/a9c30651-ccbc-4a80-900c-7880615cdf6e,,, +OCCUPATION_SOURCE,EDUCATION_DATA_ANALYST,ESCO,http://data.europa.eu/esco/occupation/d3edb8f8-3a06-47a0-8fb9-9b212c006aa2,,, +SKILL_SOURCE,JAVA,ESCO,http://data.europa.eu/esco/skill/19a8293b-8e95-4de3-983f-77484079c389,,, +SKILL_SOURCE,PYTHON,ESCO,http://data.europa.eu/esco/skill/ccd0a1d9-afda-43d9-b901-96344886e14d,,, +SKILL_SOURCE,JAVASCRIPT,ESCO,http://data.europa.eu/esco/skill/3cd569a2-4f88-4c1e-9995-8dce8c5e51a7,,, +SKILL_SOURCE,TYPESCRIPT,ESCO,http://data.europa.eu/esco/skill/867137fb-ff1b-4ca3-99f3-cb6969aa2c68,,, +SKILL_SOURCE,SQL,ESCO,http://data.europa.eu/esco/skill/598de5b0-5b58-4ea7-8058-a4bc4d18c742,,, +SKILL_SOURCE,CSHARP,ESCO,http://data.europa.eu/esco/skill/4c016b68-4116-468c-9dc6-42710c239e4a,,, +SKILL_SOURCE,HTML,ESCO,http://data.europa.eu/esco/skill/0af062de-eb43-41e9-9b96-249e2cd22d26,,, +SKILL_SOURCE,CSS,ESCO,http://data.europa.eu/esco/skill/e5d1f825-60ed-4bdd-872a-e748c387f777,,, +SKILL_SOURCE,R,ESCO,http://data.europa.eu/esco/skill/51586df8-1c46-4b47-8583-773cb63bf00b,,, +SKILL_SOURCE,MYSQL,ESCO,http://data.europa.eu/esco/skill/4da171e5-779c-4983-a76f-91c16751e99f,,, +SKILL_SOURCE,POSTGRESQL,ESCO,http://data.europa.eu/esco/skill/a8d07b5a-c1a1-42c6-9d53-db9c7a2ca996,,, +SKILL_SOURCE,STATISTICS,ESCO,http://data.europa.eu/esco/skill/7ee4c2ea-b349-4bd2-81a3-ec31475d4833,,, +SKILL_SOURCE,MACHINE_LEARNING,ESCO,http://data.europa.eu/esco/skill/8369c2d6-c100-4cf6-bd83-9668d8678433,,, +SKILL_SOURCE,FINANCIAL_ANALYSIS,ESCO,http://data.europa.eu/esco/skill/99571e68-801f-49af-a897-5f75996642e1,,, +SKILL_SOURCE,RISK_MANAGEMENT,ESCO,http://data.europa.eu/esco/skill/6eff134b-e34f-4d6e-a6e8-5e47cf2228d0,,, +SKILL_SOURCE,INSTRUCTIONAL_DESIGN,ESCO,http://data.europa.eu/esco/skill/e3145bd7-7f35-44ec-a5dc-4417c5c9ab9a,,, +SKILL_SOURCE,CURRICULUM_DEVELOPMENT,ESCO,http://data.europa.eu/esco/skill/142a5319-60cb-42a2-891f-90ba92af18b1,,, +SKILL_SOURCE,LEARNING_MANAGEMENT_SYSTEMS,ESCO,http://data.europa.eu/esco/skill/6c80d53c-d8c9-41fe-998f-091fca208834,,, +SKILL_SOURCE,EDUCATIONAL_TECHNOLOGY,ESCO,http://data.europa.eu/esco/skill/8a6394d1-f44f-4b0d-81d5-7201fc025ba8,,, +SKILL_SOURCE,COMMUNICATION,ESCO,http://data.europa.eu/esco/skill/15d76317-c71a-4fa2-aadc-2ecc34e627b7,,, +SKILL_SOURCE,CRITICAL_THINKING,ESCO,http://data.europa.eu/esco/skill/a9e6718b-b9d8-44c2-b06e-85ceed9d144c,,, +SKILL_SOURCE,COLLABORATION,ESCO,http://data.europa.eu/esco/skill/082967d4-b94c-4305-aa13-35a6aa8c5c7c,,, +SKILL_SOURCE,PROJECT_MANAGEMENT,ESCO,http://data.europa.eu/esco/skill/7111b95d-0ce3-441a-9d92-4c75d05c4388,,, +SKILL_SOURCE,AGILE,ESCO,http://data.europa.eu/esco/skill/dba46f87-0831-49cd-a1c7-340a653c0221,,, +SKILL_SOURCE,JENKINS,ESCO,http://data.europa.eu/esco/skill/f47a1998-0beb-43be-9f46-380aa4d183da,,, +SKILL_SOURCE,REACT,O*NET,15-1254.00|React|2.E.7.h,,, +SKILL_SOURCE,VUE_JS,O*NET,15-1254.00|Vue.js|2.E.7.h,,, +SKILL_SOURCE,NODE_JS,O*NET,15-1254.00|Node.js|2.E.7.h,,, +SKILL_SOURCE,SPRING_BOOT,O*NET,15-1252.00|Spring Boot|2.E.10.a,,, +SKILL_SOURCE,DJANGO,O*NET,15-1252.00|Django|2.E.7.h,,, +SKILL_SOURCE,FLASK,O*NET,15-2051.00|Flask|2.E.7.b,,, +SKILL_SOURCE,GIT,O*NET,15-1252.00|Git|2.E.5.c,,, +SKILL_SOURCE,DOCKER,O*NET,15-1252.00|Docker|2.E.10.a,,, +SKILL_SOURCE,KUBERNETES,O*NET,15-1252.00|Kubernetes|2.E.10.a,,, +SKILL_SOURCE,GRADLE,O*NET,15-1252.00|Gradle|2.E.7.e,,, +SKILL_SOURCE,MONGODB,O*NET,15-1252.00|MongoDB|2.E.6.d,,, +SKILL_SOURCE,REDIS,O*NET,15-1252.00|Redis|2.E.6.d,,, +SKILL_SOURCE,SELENIUM,O*NET,15-1253.00|Selenium|2.E.7.f,,, +SKILL_SOURCE,JUNIT,O*NET,15-1253.00|JUnit|2.E.7.f,,, +SKILL_SOURCE,POSTMAN,O*NET,15-1253.00|Postman|2.E.13.c,,, +SKILL_SOURCE,PANDAS,O*NET,15-2051.00|pandas|2.E.6.f,,, +SKILL_SOURCE,NUMPY,O*NET,15-2051.00|NumPy|2.E.6.f,,, +SKILL_SOURCE,TABLEAU,O*NET,15-2051.00|Tableau|2.E.6.m,,, +SKILL_SOURCE,POWER_BI,O*NET,15-2051.00|Microsoft Power BI|2.E.6.m,,, +SKILL_SOURCE,MICROSOFT_EXCEL,O*NET,15-2051.00|Microsoft Excel|2.E.4.i,,, +SKILL_SOURCE,TENSORFLOW,O*NET,15-2051.00|TensorFlow|2.E.9.d,,, +SKILL_SOURCE,PYTORCH,O*NET,15-2051.00|PyTorch|2.E.6.f,,, +SKILL_SOURCE,SCIKIT_LEARN,O*NET,15-2051.00|Scikit-learn|2.E.7.b,,, +SKILL_SOURCE,APACHE_SPARK,O*NET,15-2051.00|Apache Spark|2.E.6.m,,, +SKILL_SOURCE,FIGMA,O*NET,15-1254.00|Figma|2.E.7.d,,, +SKILL_SOURCE,BASH,O*NET,15-1252.00|Bash|2.E.13.c,,, +SKILL_SOURCE,GO,O*NET,15-1252.00|Go|2.E.7.b,,, +SKILL_SOURCE,CPP,O*NET,15-1252.00|C++|2.E.7.e,,, +SKILL_SOURCE,GOOGLE_CHROME,O*NET,17-2051.02|Google Chrome|2.E.10.e,,, +SKILL_SOURCE,DEBUGGING_SOFTWARE,O*NET,17-2071.00|Debugging software|2.E.7.f,,, +SKILL_SOURCE,APACHE_KAFKA,O*NET,15-1299.07|Apache Kafka|2.E.7.b,,, +SKILL_SOURCE,SONARQUBE,O*NET,15-1252.00|SonarQube|2.E.7.f,,, +SKILL_SOURCE,CHEF,O*NET,15-1255.00|Chef|2.E.7.a,,, +SKILL_SOURCE,GITLAB,O*NET,15-1299.08|GitLab|2.E.10.a,,, +SKILL_SOURCE,NAGIOS,O*NET,11-3021.00|Nagios|2.E.11.a,,, +SKILL_SOURCE,NMAP,O*NET,13-1199.07|Nmap|2.E.14.d,,, +SKILL_SOURCE,BLOOMBERG_PROFESSIONAL,O*NET,41-3031.00|Bloomberg Professional|2.E.2.d,,, +SKILL_SOURCE,MOODLE,O*NET,25-2022.00|Moodle|2.E.8.b,,, +SKILL_SOURCE,ARTICULATE_STORYLINE,O*NET,25-9031.00|Articulate Storyline|2.E.8.b,,, +SKILL_SOURCE,MICROSOFT_POWERPOINT,O*NET,27-4012.00|Microsoft PowerPoint|2.E.4.f,,, +SKILL_SOURCE,AXURE_RP,O*NET,15-1255.00|Axure RP|2.E.4.g,,, +SKILL_SOURCE,ADOBE_PHOTOSHOP,O*NET,19-3092.00|Adobe Photoshop|2.E.4.b,,, +SKILL_SOURCE,APACHE_JMETER,O*NET,15-1253.00|Apache JMeter|2.E.7.e,,, +SKILL_SOURCE,ADOBE_ACROBAT,O*NET,11-1011.00|Adobe Acrobat|2.E.5.b,,, +SKILL_SOURCE,MICROSOFT_AZURE,O*NET,41-4011.00|Microsoft Azure software|2.E.7.b,,, +SKILL_SOURCE,AWS,O*NET,11-1021.00|Amazon Web Services AWS software|2.E.6.f,,, +SKILL_SOURCE,WIRESHARK,O*NET,15-1241.01|Wireshark|2.E.11.a,,, +RELATION,FRONTEND_DEVELOPER|JAVASCRIPT,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|TYPESCRIPT,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|HTML,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|CSS,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|REACT,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|VUE_JS,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|NODE_JS,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|GRADLE,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|GOOGLE_CHROME,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|DEBUGGING_SOFTWARE,O*NET,15-1254.00,4.A.3.b.1,4.88,1 +RELATION,FRONTEND_DEVELOPER|SELENIUM,O*NET,15-1254.00,4.A.2.a.3,3.33,1 +RELATION,FRONTEND_DEVELOPER|JUNIT,O*NET,15-1254.00,4.A.2.a.3,3.33,1 +RELATION,FRONTEND_DEVELOPER|GIT,O*NET,15-1254.00,4.A.4.a.2,3.88,1 +RELATION,FRONTEND_DEVELOPER|AGILE,O*NET,15-1254.00,4.A.4.a.2,3.88,1 +RELATION,BACKEND_DEVELOPER|JAVA,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|PYTHON,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|SPRING_BOOT,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|DJANGO,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|MYSQL,O*NET,15-1252.00,4.A.2.a.2,4.38,1 +RELATION,BACKEND_DEVELOPER|POSTGRESQL,O*NET,15-1252.00,4.A.2.a.2,4.38,1 +RELATION,BACKEND_DEVELOPER|REDIS,O*NET,15-1252.00,4.A.2.a.2,4.38,1 +RELATION,BACKEND_DEVELOPER|APACHE_KAFKA,O*NET,15-1252.00,4.A.2.a.2,4.38,1 +RELATION,BACKEND_DEVELOPER|DOCKER,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|KUBERNETES,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|JUNIT,O*NET,15-1252.00,4.A.2.a.3,3.94,1 +RELATION,BACKEND_DEVELOPER|POSTMAN,O*NET,15-1252.00,4.A.2.a.3,3.94,1 +RELATION,BACKEND_DEVELOPER|GIT,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|GRADLE,O*NET,15-1252.00,4.A.3.b.1,4.61,1 +RELATION,BACKEND_DEVELOPER|NMAP,O*NET,15-1252.00,2.C.8.a,2.57,1 +RELATION,BACKEND_DEVELOPER|SONARQUBE,O*NET,15-1252.00,2.C.8.a,2.57,1 +RELATION,QA_ENGINEER|JAVA,O*NET,15-1253.00,4.A.3.b.1,4.94,1 +RELATION,QA_ENGINEER|PYTHON,O*NET,15-1253.00,4.A.3.b.1,4.94,1 +RELATION,QA_ENGINEER|POSTMAN,O*NET,15-1253.00,4.A.2.a.3,4.24,1 +RELATION,QA_ENGINEER|CRITICAL_THINKING,O*NET,15-1253.00,4.A.2.a.3,4.24,1 +RELATION,QA_ENGINEER|SELENIUM,O*NET,15-1253.00,4.A.3.b.1,4.94,1 +RELATION,QA_ENGINEER|JUNIT,O*NET,15-1253.00,4.A.3.b.1,4.94,1 +RELATION,QA_ENGINEER|SQL,O*NET,15-1253.00,4.A.2.a.2,4.38,1 +RELATION,QA_ENGINEER|MYSQL,O*NET,15-1253.00,4.A.2.a.2,4.38,1 +RELATION,QA_ENGINEER|APACHE_JMETER,O*NET,15-1253.00,4.A.1.b.3,2.94,1 +RELATION,QA_ENGINEER|BASH,O*NET,15-1253.00,4.A.1.b.3,2.94,1 +RELATION,QA_ENGINEER|JENKINS,O*NET,15-1253.00,4.A.3.b.1,4.94,1 +RELATION,QA_ENGINEER|GITLAB,O*NET,15-1253.00,4.A.3.b.1,4.94,1 +RELATION,DEVOPS_ENGINEER|PYTHON,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|BASH,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|AWS,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|MICROSOFT_AZURE,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|DOCKER,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|KUBERNETES,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|JENKINS,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|GITLAB,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|CHEF,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|GIT,O*NET,15-1244.00,4.A.3.b.1,4.83,1 +RELATION,DEVOPS_ENGINEER|NAGIOS,O*NET,15-1244.00,4.A.1.a.2,3.60,1 +RELATION,DEVOPS_ENGINEER|WIRESHARK,O*NET,15-1244.00,4.A.1.a.2,3.60,1 +RELATION,DEVOPS_ENGINEER|NMAP,O*NET,15-1244.00,2.C.8.a,2.34,1 +RELATION,DEVOPS_ENGINEER|SONARQUBE,O*NET,15-1244.00,2.C.8.a,2.34,1 +RELATION,DEVOPS_ENGINEER|MYSQL,O*NET,15-1244.00,4.A.2.a.2,4.03,1 +RELATION,DEVOPS_ENGINEER|POSTGRESQL,O*NET,15-1244.00,4.A.2.a.2,4.03,1 +RELATION,FINANCIAL_DATA_ANALYST|PYTHON,O*NET,13-2031.00,4.A.3.b.1,4.48,1 +RELATION,FINANCIAL_DATA_ANALYST|R,O*NET,13-2031.00,4.A.3.b.1,4.48,1 +RELATION,FINANCIAL_DATA_ANALYST|PANDAS,O*NET,13-2031.00,4.A.2.a.4,4.01,1 +RELATION,FINANCIAL_DATA_ANALYST|TABLEAU,O*NET,13-2031.00,4.A.2.a.4,4.01,1 +RELATION,FINANCIAL_DATA_ANALYST|SQL,O*NET,13-2031.00,4.A.2.a.2,4.44,1 +RELATION,FINANCIAL_DATA_ANALYST|POSTGRESQL,O*NET,13-2031.00,4.A.2.a.2,4.44,1 +RELATION,FINANCIAL_DATA_ANALYST|STATISTICS,O*NET,13-2031.00,2.C.4.a,4.03,1 +RELATION,FINANCIAL_DATA_ANALYST|MACHINE_LEARNING,O*NET,13-2031.00,2.C.4.a,4.03,1 +RELATION,FINANCIAL_DATA_ANALYST|BLOOMBERG_PROFESSIONAL,O*NET,13-2031.00,4.A.3.b.1,4.48,1 +RELATION,FINANCIAL_DATA_ANALYST|MICROSOFT_EXCEL,O*NET,13-2031.00,4.A.3.b.1,4.48,1 +RELATION,FINANCIAL_DATA_ANALYST|FINANCIAL_ANALYSIS,O*NET,13-2031.00,2.C.1.c,4.42,1 +RELATION,FINANCIAL_DATA_ANALYST|RISK_MANAGEMENT,O*NET,13-2031.00,2.C.1.c,4.42,1 +RELATION,FINANCIAL_DATA_ANALYST|GIT,O*NET,13-2031.00,4.A.3.b.1,4.48,1 +RELATION,FINANCIAL_DATA_ANALYST|POWER_BI,O*NET,13-2031.00,4.A.3.b.1,4.48,1 +RELATION,RISK_MODELING_ENGINEER|PYTHON,O*NET,13-2053.00,4.A.3.b.1,4.41,1 +RELATION,RISK_MODELING_ENGINEER|R,O*NET,13-2053.00,4.A.3.b.1,4.41,1 +RELATION,RISK_MODELING_ENGINEER|STATISTICS,O*NET,13-2053.00,2.C.4.a,3.27,1 +RELATION,RISK_MODELING_ENGINEER|MACHINE_LEARNING,O*NET,13-2053.00,2.C.4.a,3.27,1 +RELATION,RISK_MODELING_ENGINEER|RISK_MANAGEMENT,O*NET,13-2053.00,4.A.2.a.4,4.43,1 +RELATION,RISK_MODELING_ENGINEER|CRITICAL_THINKING,O*NET,13-2053.00,4.A.2.a.4,4.43,1 +RELATION,RISK_MODELING_ENGINEER|TENSORFLOW,O*NET,13-2053.00,4.A.3.b.1,4.41,1 +RELATION,RISK_MODELING_ENGINEER|SCIKIT_LEARN,O*NET,13-2053.00,4.A.3.b.1,4.41,1 +RELATION,RISK_MODELING_ENGINEER|SQL,O*NET,13-2053.00,4.A.2.a.2,4.13,1 +RELATION,RISK_MODELING_ENGINEER|APACHE_SPARK,O*NET,13-2053.00,4.A.2.a.2,4.13,1 +RELATION,RISK_MODELING_ENGINEER|JUNIT,O*NET,13-2053.00,4.A.2.a.3,3.90,1 +RELATION,RISK_MODELING_ENGINEER|SONARQUBE,O*NET,13-2053.00,4.A.2.a.3,3.90,1 +RELATION,RISK_MODELING_ENGINEER|DOCKER,O*NET,13-2053.00,4.A.3.b.1,4.41,1 +RELATION,RISK_MODELING_ENGINEER|GIT,O*NET,13-2053.00,4.A.3.b.1,4.41,1 +RELATION,QUANTITATIVE_DEVELOPER|PYTHON,O*NET,13-2099.01,4.A.3.b.1,4.68,1 +RELATION,QUANTITATIVE_DEVELOPER|CPP,O*NET,13-2099.01,4.A.3.b.1,4.68,1 +RELATION,QUANTITATIVE_DEVELOPER|GO,O*NET,13-2099.01,4.A.3.b.1,4.68,1 +RELATION,QUANTITATIVE_DEVELOPER|NUMPY,O*NET,13-2099.01,4.A.3.b.1,4.68,1 +RELATION,QUANTITATIVE_DEVELOPER|SCIKIT_LEARN,O*NET,13-2099.01,4.A.3.b.1,4.68,1 +RELATION,QUANTITATIVE_DEVELOPER|BLOOMBERG_PROFESSIONAL,O*NET,13-2099.01,2.C.1.c,4.20,1 +RELATION,QUANTITATIVE_DEVELOPER|FINANCIAL_ANALYSIS,O*NET,13-2099.01,2.C.1.c,4.20,1 +RELATION,QUANTITATIVE_DEVELOPER|POSTGRESQL,O*NET,13-2099.01,4.A.2.a.2,4.50,1 +RELATION,QUANTITATIVE_DEVELOPER|SQL,O*NET,13-2099.01,4.A.2.a.2,4.50,1 +RELATION,QUANTITATIVE_DEVELOPER|APACHE_SPARK,O*NET,13-2099.01,4.A.1.b.3,3.53,1 +RELATION,QUANTITATIVE_DEVELOPER|APACHE_KAFKA,O*NET,13-2099.01,4.A.1.b.3,3.53,1 +RELATION,QUANTITATIVE_DEVELOPER|JUNIT,O*NET,13-2099.01,4.A.2.a.3,3.05,1 +RELATION,QUANTITATIVE_DEVELOPER|APACHE_JMETER,O*NET,13-2099.01,4.A.2.a.3,3.05,1 +RELATION,QUANTITATIVE_DEVELOPER|DOCKER,O*NET,13-2099.01,4.A.3.b.1,4.68,1 +RELATION,QUANTITATIVE_DEVELOPER|KUBERNETES,O*NET,13-2099.01,4.A.3.b.1,4.68,1 +RELATION,EDUCATION_PRODUCT_MANAGER|INSTRUCTIONAL_DESIGN,O*NET,13-1082.00,2.C.6,2.52,1 +RELATION,EDUCATION_PRODUCT_MANAGER|CURRICULUM_DEVELOPMENT,O*NET,13-1082.00,2.C.6,2.52,1 +RELATION,EDUCATION_PRODUCT_MANAGER|AXURE_RP,O*NET,13-1082.00,4.A.2.a.4,3.94,1 +RELATION,EDUCATION_PRODUCT_MANAGER|POWER_BI,O*NET,13-1082.00,4.A.2.a.4,3.94,1 +RELATION,EDUCATION_PRODUCT_MANAGER|PROJECT_MANAGEMENT,O*NET,13-1082.00,4.A.2.b.4,3.73,1 +RELATION,EDUCATION_PRODUCT_MANAGER|AGILE,O*NET,13-1082.00,4.A.2.b.4,3.73,1 +RELATION,EDUCATION_PRODUCT_MANAGER|TABLEAU,O*NET,13-1082.00,4.A.2.a.4,3.94,1 +RELATION,EDUCATION_PRODUCT_MANAGER|PYTHON,O*NET,13-1082.00,4.A.2.a.4,3.94,1 +RELATION,EDUCATION_PRODUCT_MANAGER|LEARNING_MANAGEMENT_SYSTEMS,O*NET,13-1082.00,4.A.3.b.1,4.45,1 +RELATION,EDUCATION_PRODUCT_MANAGER|MOODLE,O*NET,13-1082.00,4.A.3.b.1,4.45,1 +RELATION,EDUCATION_PRODUCT_MANAGER|GIT,O*NET,13-1082.00,4.A.4.a.2,4.73,1 +RELATION,EDUCATION_PRODUCT_MANAGER|MICROSOFT_EXCEL,O*NET,13-1082.00,4.A.4.a.2,4.73,1 +RELATION,INSTRUCTIONAL_DESIGNER|INSTRUCTIONAL_DESIGN,O*NET,25-9031.00,2.C.6,4.78,1 +RELATION,INSTRUCTIONAL_DESIGNER|CURRICULUM_DEVELOPMENT,O*NET,25-9031.00,2.C.6,4.78,1 +RELATION,INSTRUCTIONAL_DESIGNER|LEARNING_MANAGEMENT_SYSTEMS,O*NET,25-9031.00,4.A.3.b.1,4.45,1 +RELATION,INSTRUCTIONAL_DESIGNER|MOODLE,O*NET,25-9031.00,4.A.3.b.1,4.45,1 +RELATION,INSTRUCTIONAL_DESIGNER|ARTICULATE_STORYLINE,O*NET,25-9031.00,4.A.2.b.2,3.72,1 +RELATION,INSTRUCTIONAL_DESIGNER|FIGMA,O*NET,25-9031.00,4.A.2.b.2,3.72,1 +RELATION,INSTRUCTIONAL_DESIGNER|EDUCATIONAL_TECHNOLOGY,O*NET,25-9031.00,4.A.2.a.3,3.83,1 +RELATION,INSTRUCTIONAL_DESIGNER|CRITICAL_THINKING,O*NET,25-9031.00,4.A.2.a.3,3.83,1 +RELATION,INSTRUCTIONAL_DESIGNER|STATISTICS,O*NET,25-9031.00,4.A.2.a.4,4.19,1 +RELATION,INSTRUCTIONAL_DESIGNER|MICROSOFT_EXCEL,O*NET,25-9031.00,4.A.2.a.4,4.19,1 +RELATION,INSTRUCTIONAL_DESIGNER|ADOBE_ACROBAT,O*NET,25-9031.00,4.A.2.a.3,3.83,1 +RELATION,INSTRUCTIONAL_DESIGNER|COMMUNICATION,O*NET,25-9031.00,4.A.2.a.3,3.83,1 +RELATION,EDUCATION_DATA_ANALYST|PYTHON,O*NET,15-2051.00,4.A.3.b.1,4.75,1 +RELATION,EDUCATION_DATA_ANALYST|R,O*NET,15-2051.00,4.A.3.b.1,4.75,1 +RELATION,EDUCATION_DATA_ANALYST|TABLEAU,O*NET,15-2051.00,4.A.2.a.4,4.83,1 +RELATION,EDUCATION_DATA_ANALYST|POWER_BI,O*NET,15-2051.00,4.A.2.a.4,4.83,1 +RELATION,EDUCATION_DATA_ANALYST|SQL,O*NET,15-2051.00,4.A.2.a.2,4.71,1 +RELATION,EDUCATION_DATA_ANALYST|POSTGRESQL,O*NET,15-2051.00,4.A.2.a.2,4.71,1 +RELATION,EDUCATION_DATA_ANALYST|STATISTICS,O*NET,15-2051.00,2.C.4.a,4.52,1 +RELATION,EDUCATION_DATA_ANALYST|MACHINE_LEARNING,O*NET,15-2051.00,2.C.4.a,4.52,1 +RELATION,EDUCATION_DATA_ANALYST|CURRICULUM_DEVELOPMENT,O*NET,15-2051.00,2.C.6,2.28,1 +RELATION,EDUCATION_DATA_ANALYST|EDUCATIONAL_TECHNOLOGY,O*NET,15-2051.00,2.C.6,2.28,1 +RELATION,EDUCATION_DATA_ANALYST|MICROSOFT_EXCEL,O*NET,15-2051.00,4.A.3.b.1,4.75,1 +RELATION,EDUCATION_DATA_ANALYST|MICROSOFT_POWERPOINT,O*NET,15-2051.00,4.A.3.b.1,4.75,1 +RELATION,FRONTEND_DEVELOPER|FIGMA,O*NET,,,,0 +RELATION,INSTRUCTIONAL_DESIGNER|ADOBE_PHOTOSHOP,O*NET,,,,0 diff --git a/specs/learning-path/spec.md b/specs/learning-path/spec.md new file mode 100644 index 0000000..613d2ca --- /dev/null +++ b/specs/learning-path/spec.md @@ -0,0 +1,88 @@ +# learning path Specification + +## Purpose + +定义基于缺失 TOP10 技能的确定性学习路径生成行为。 + +## Requirements + +### Requirement: Learning path consumes unmatched TOP10 skills + +学习路径接口 MUST 复用个人技能匹配结果，并只包含当前 TOP10 中未命中的技能；不得包含已命中技能或 TOP10 之外的技能。 + +#### Scenario: Student has skill gaps + +- **WHEN** 当前 TOP10 中部分技能未命中 +- **THEN** 学习路径 MUST 包含全部未命中技能且不含已命中技能 + +#### Scenario: Student has no skill gaps + +- **WHEN** 当前 TOP10 全部命中 +- **THEN** 学习路径 MUST 返回空列表和总数 0 + +#### Scenario: Student has no matching skill evidence + +- **WHEN** 学生没有技能数据 +- **THEN** 学习路径 MUST 包含当前 TOP10 的全部技能 + +### Requirement: Learning priority follows heat order + +学习路径项 MUST 按与 TOP10 相同的热度顺序排列，并从 1 开始连续编号；空热度技能继续遵守词云的末位排序规则。 + +#### Scenario: Multiple missing skills are ordered + +- **WHEN** 缺失技能包含多个热度不同的技能 +- **THEN** 热度最高的技能 MUST 获得优先级 1，后续技能依次编号 + +#### Scenario: Missing skills have equal heat + +- **WHEN** 缺失技能热度相同 +- **THEN** 优先级 MUST 沿用词云的稳定次级排序，重复请求结果一致 + +#### Scenario: Only one skill is missing + +- **WHEN** 当前 TOP10 只有一项未命中 +- **THEN** 该技能 MUST 返回优先级 1 + +### Requirement: Learning suggestions are deterministic and industry-neutral + +每项学习建议 MUST 由后端按技能分类的通用模板生成，不调用 AI、知识库问答或外部课程服务；建议不得包含课程链接、讲师、价格、学习时长或完成度。 + +#### Scenario: Programming language gap is returned + +- **WHEN** 缺失技能属于编程语言分类 +- **THEN** 建议 MUST 描述基础语法学习和编码练习方向 + +#### Scenario: Framework gap is returned + +- **WHEN** 缺失技能属于框架分类 +- **THEN** 建议 MUST 描述核心概念学习和小型项目实践方向 + +#### Scenario: Tool gap is returned + +- **WHEN** 缺失技能属于工具分类 +- **THEN** 建议 MUST 描述安装配置和工程实践方向 + +#### Scenario: Unknown category is returned + +- **WHEN** 缺失技能无法匹配特定模板分类 +- **THEN** 系统 MUST 返回一条行业无关的通用学习建议 + +### Requirement: Learning path response is actionable and minimal + +每个学习路径项 MUST 返回技能 ID、技能名称、技能分类、优先级和简化学习建议；响应 MUST 不包含个人隐私、简历原文或模型推理内容。 + +#### Scenario: A learning item is generated + +- **WHEN** 存在未命中技能 +- **THEN** 每个路径项 MUST 同时具备技能身份、分类、连续优先级和建议文本 + +#### Scenario: Repeated request is made + +- **WHEN** 学生资料、词库数据和筛选条件均未变化 +- **THEN** 接口 MUST 返回相同顺序、相同优先级和相同建议 + +#### Scenario: Response contains no private profile data + +- **WHEN** 学习路径成功返回 +- **THEN** 响应 MUST 不含手机号、邮箱、简历文件、原始 `skills_json` 或其他无关个人数据 diff --git a/specs/skill-cloud/spec.md b/specs/skill-cloud/spec.md new file mode 100644 index 0000000..5201287 --- /dev/null +++ b/specs/skill-cloud/spec.md @@ -0,0 +1,102 @@ +# skill cloud Specification + +## Purpose + +定义行业技能词云查询、筛选、热度排序与来源追溯行为。 + +## Requirements + +### Requirement: STUDENT-only skill cloud access + +系统 MUST 只允许已登录 STUDENT 调用技能词云接口，并从 JWT 解析当前用户，不接受客户端传入学生身份参数。 + +#### Scenario: Anonymous request is rejected + +- **WHEN** 未携带有效 JWT 请求 `GET /api/v1/skills/word-cloud` +- **THEN** 系统 MUST 返回 401，且不得返回任何词库数据 + +#### Scenario: Company or administrator request is rejected + +- **WHEN** COMPANY 或 ADMIN 用户携带有效 JWT 请求技能词云 +- **THEN** 系统 MUST 返回 403，且不得返回任何词库数据 + +#### Scenario: Student request is accepted + +- **WHEN** STUDENT 用户携带有效 JWT 请求技能词云 +- **THEN** 系统 MUST 按筛选条件返回词库数据 + +### Requirement: Skill cloud filter validation + +接口 MUST 要求 `industryId` 和 `occupationId` 为必填正整数，`categoryId` 为可选正整数；行业与职位必须存在且互为关联，技能分类存在时必须属于该职位。 + +#### Scenario: Required filters are missing + +- **WHEN** 请求未传 `industryId` 或 `occupationId` +- **THEN** 系统 MUST 返回 400，并说明缺失的筛选参数 + +#### Scenario: Occupation does not belong to industry + +- **WHEN** 请求传入存在但不属于目标行业的职位 +- **THEN** 系统 MUST 返回 400，且不得返回技能数据 + +#### Scenario: Category does not belong to occupation + +- **WHEN** 请求传入存在但不属于目标职位的技能分类 +- **THEN** 系统 MUST 返回 400，且不得返回技能数据 + +#### Scenario: Category is omitted + +- **WHEN** 请求只传行业和职位 +- **THEN** 系统 MUST 返回该职位下所有技能分类的技能 + +### Requirement: Complete ranked skill cloud + +系统 MUST 返回当前筛选范围内全部技能，并按 `heat` 降序排列；`heat` 为空时排在非空值之后。同 `heat` 或同为空值时，系统 MUST 使用稳定的中文技能名称和技能 ID 作为次级排序条件。 + +#### Scenario: Skills are ordered by heat + +- **WHEN** 筛选结果存在多个带 O*NET Importance 的技能 +- **THEN** 返回列表 MUST 按数值从高到低排列 + +#### Scenario: Skills without heat are retained + +- **WHEN** 筛选结果包含无 O*NET Importance 的 ESCO 技能 +- **THEN** 这些技能 MUST 保留在列表中，`heat` 为 null，并排在所有非 null 热度技能之后 + +#### Scenario: Equal heat has deterministic order + +- **WHEN** 两个技能具有相同 `heat` +- **THEN** 系统 MUST 按稳定次级规则返回一致顺序，重复请求不得随机变化 + +#### Scenario: Empty scope returns empty list + +- **WHEN** 合法筛选范围内没有技能 +- **THEN** 系统 MUST 返回成功响应、空技能列表和总数 0 + +### Requirement: TOP10 is derived from the skill cloud + +系统 MUST NOT 为 TOP10 维护独立排序或独立数据源；前端必须能够直接使用词云返回列表的前十条得到 TOP10。 + +#### Scenario: Scope has at least ten skills + +- **WHEN** 词云结果包含至少十条技能 +- **THEN** 前十条 MUST 与同一筛选条件下的热门技能 TOP10 一致 + +#### Scenario: Scope has fewer than ten skills + +- **WHEN** 词云结果少于十条技能 +- **THEN** 全部结果 MUST 构成实际 TOP10 集合 + +### Requirement: Skill cloud response exposes source-linked metadata + +每个技能项 MUST 至少返回技能 ID、中文名称、技能分类 ID、技能分类名称、`heat` 和稳定排名。技能仍可通过 ID 追溯至 ESCO/O*NET 来源。 + +#### Scenario: Skill item is returned + +- **WHEN** 查询返回任一技能 +- **THEN** 响应 MUST 包含该技能的身份、分类、热度和排名字段 + +#### Scenario: Response is wrapped consistently + +- **WHEN** 请求成功 +- **THEN** 响应 MUST 使用项目统一 `ApiResponse` 结构，并在 `data` 中返回筛选条件和技能列表 diff --git a/specs/skill-match/spec.md b/specs/skill-match/spec.md new file mode 100644 index 0000000..823208d --- /dev/null +++ b/specs/skill-match/spec.md @@ -0,0 +1,102 @@ +# skill match Specification + +## Purpose + +定义个人技能与当前热门 TOP10 的确定性匹配行为。 + +## Requirements + +### Requirement: Personal matching uses the current ranked TOP10 + +个人技能匹配接口 MUST 复用技能词云相同的筛选参数、数据范围和热度排序；匹配集合 MUST 是当前筛选结果的前 `min(10, 实际技能数)` 项。 + +#### Scenario: Filtered result has at least ten skills + +- **WHEN** 当前筛选返回十条或更多技能 +- **THEN** 匹配集合 MUST 恰好包含热度最高的十条技能 + +#### Scenario: Filtered result has fewer than ten skills + +- **WHEN** 当前筛选返回六条技能 +- **THEN** 匹配集合 MUST 包含全部六条技能，分母 MUST 为 6 + +#### Scenario: Filtered result is empty + +- **WHEN** 当前合法筛选范围内没有技能 +- **THEN** 匹配度 MUST 返回 0，命中和缺失列表 MUST 均为空 + +### Requirement: Student skills are read from the authenticated profile + +系统 MUST 从当前登录 STUDENT 的 `student_profiles.skills_json` 读取技能；不得接受客户端另行提交的技能列表作为个人匹配依据。 + +#### Scenario: Student profile has valid skills + +- **WHEN** 当前学生存在合法技能数组 +- **THEN** 系统 MUST 使用该数组计算匹配结果 + +#### Scenario: Student profile is absent or has no skills + +- **WHEN** 当前学生没有资料记录、技能字段为空或技能数组为空 +- **THEN** 匹配度 MUST 返回 0，命中列表为空，缺失列表为当前 TOP10 + +#### Scenario: Skills JSON is malformed + +- **WHEN** 当前学生技能字段无法解析为字符串数组 +- **THEN** 系统 MUST 按无技能处理，不得导致 500 响应或泄露原始字段内容 + +### Requirement: Skill matching normalizes aliases + +系统 MUST 使用技能标准和别名进行匹配，归一化至少覆盖首尾空白、大小写、连续空格和常见标点差异；每个标准技能命中一次即可。 + +#### Scenario: Alias matches canonical skill + +- **WHEN** 学生技能为 `SpringBoot`，TOP10 标准技能为 `Spring Boot` 且存在对应别名 +- **THEN** 系统 MUST 将该技能计为命中 + +#### Scenario: Case and spacing differ + +- **WHEN** 学生技能与标准技能的字母大小写或空格数量不同 +- **THEN** 系统 MUST 在归一化后判断为同一技能 + +#### Scenario: Duplicate student skills exist + +- **WHEN** 学生技能数组包含多个指向同一标准技能的别名 +- **THEN** 系统 MUST 只计一次命中 + +### Requirement: Match score uses the actual TOP10 size + +匹配度 MUST 使用公式 `round(命中数 × 100 / min(10, 实际技能数))` 计算，并返回 0 到 100 的整数。实际技能数为 0 时，匹配度 MUST 为 0。 + +#### Scenario: All skills match + +- **WHEN** 当前 TOP10 有 8 项，学生命中全部 8 项 +- **THEN** 匹配度 MUST 返回 100 + +#### Scenario: Partial match is rounded + +- **WHEN** 当前 TOP10 有 3 项，学生命中 1 项 +- **THEN** 匹配度 MUST 返回 33 + +#### Scenario: No skill matches + +- **WHEN** 当前 TOP10 非空且学生没有命中任何技能 +- **THEN** 匹配度 MUST 返回 0 + +### Requirement: Match response separates matched and missing skills + +响应 MUST 返回匹配度、命中数量、实际匹配集合大小、命中技能列表和缺失技能列表；缺失技能 MUST 保持 TOP10 的热度排序。 + +#### Scenario: Some skills are matched + +- **WHEN** 学生命中 TOP10 的部分技能 +- **THEN** 命中列表和缺失列表 MUST 分别返回对应技能，且并集等于 TOP10 + +#### Scenario: All skills are matched + +- **WHEN** 学生命中 TOP10 的全部技能 +- **THEN** 缺失技能列表 MUST 为空，匹配度 MUST 为 100 + +#### Scenario: No skills are matched + +- **WHEN** 学生没有命中 TOP10 的任何技能 +- **THEN** 命中技能列表 MUST 为空，缺失列表 MUST 等于 TOP10