CONSORT: ! reconstruct the LinkedIn/World Bank skill taxonomy

CONSORT Structured English for AI Specification: https://github.com/mwherman2000/Consort/blob/main/Consort%200.12%20system%20prompt.txt

! reconstruct the LinkedIn/World Bank skill taxonomy

@ lead taxonomy architect and research orchestrator

# objective

Build a verified machine-readable representation of the LinkedIn/World Bank skill taxonomy:

Broad Category → Skill Group → LinkedIn Skill

The historical World Bank/LinkedIn taxonomy and the current LinkedIn Skills Graph must remain separate.

# known source state (from prior run — read before executing, do not silently re-derive)

• Primary source confirmed: World Bank Group | LinkedIn Data Insights: Jobs, Skills and
  Migration Trends — Methodology & Validation Results (Zhu, Fritzler, Orlowski; WBG/LinkedIn,
  Nov 2018). Appendix F: Skill Group Classification, pp. 86–94.
  https://documents1.worldbank.org/curated/en/827991542143093021/pdf/World-Bank-Group-LinkedIn-Data-Insights-Jobs-Skills-and-Migration-Trends-Methodology-and-Validation-Results.pdf
  Automated web-fetch of this URL truncates before page 86 and cannot reach Appendix F on its
  own — if the user has not attached the full PDF in this run, ask for it before starting
  phase 2 rather than proceeding on a partial fetch.
• Companion source, “Reference: Skill Group Definitions” (standalone PDF), World Bank Data
  Catalog dataset 0038027 (“Skills | LinkedIn Data”), last updated Sept 22, 2020:
  https://datalakeesouoprod.blob.core.windows.net/data/ddh/data/ddh-published/0038027/1/DR0046193/skill-group-definitions.pdf
  This is likely the more authoritative and more current version of the same table, and is
  the most plausible place a “Broad Category” tier (see below) could actually be defined.
  It has been blocked by bot detection on every automated fetch attempt so far. This run
  MUST re-attempt the fetch in phase 1. If it is still blocked, phase 1 MUST explicitly ask
  the user to manually download and upload it before phase 2 proceeds — do not silently drop
  this source and do not fabricate a category tier in its absence.
• Appendix F, as currently confirmed, is a TWO-level taxonomy only: Skill Group → sample
  Detailed Skills. It defines no Broad Category tier. Do not invent one if the companion
  source above remains unavailable — report the gap instead (see phase 5 and the final
  report’s “five-category mappings” line).
• Appendix F prints only a SAMPLE of skills per group (previously observed: ~9.6 samples/
  group average, ~2,352 sample skill mentions across 246 groups), not the full ~10,000-skill
  membership the report’s own body text (Section V, p. 59) references. Every phase-2 record
  must state this per group — not just once in a README.

$ verification-first
$ preserve source provenance
$ never fabricate missing information
$ preserve taxonomy versions
$ preserve multiple skill-group memberships
$ distinguish source facts from inference
$ show intermediate stages

# phase 1 — discover authoritative taxonomy structure

| discover:

identify the authoritative World Bank/LinkedIn documents containing:

• Skill Group Definitions
• Appendix F
• skill-group/skill mappings
• broad skill categories
• taxonomy version/date
• methodology

Re-attempt fetching the “Skill Group Definitions” companion PDF (see “known source state”
above) and report pass/fail explicitly. If blocked, ask the user for a manual upload before
continuing to phase 2.

return:

groups
categories
taxonomy_versions
sources (including explicit fetch status for each — retrieved / blocked / not attempted)

# phase 2 — dynamically extract every skill group

| extract:

^ for-each group in discover.groups:

! extract and verify every LinkedIn skill belonging to %group%
# retrieve the original source material for %group%
# extract the exact skill names
# preserve source spelling and capitalization
# record source document and page — per skill-group entry, not a blanket page range for
  the whole appendix, when the source's page-break markers make per-entry attribution
  possible
# identify the taxonomy version
# identify the broad category when explicitly supported by a source; when it is not
  (e.g. Appendix F alone), the field is populated with "not present in source" rather
  than omitted
$ do not infer membership
$ do not invent missing skills
$ do not silently normalize names
$ preserve duplicate or multi-group relationships
$ label skills[] as a SAMPLE, not exhaustive membership, unless the source is confirmed
  to be a complete crosswalk
% return:
  skill_group
  skill_group_definition (state "not defined in source" rather than omitting, if absent)
  top_level_category
  skills[]
  taxonomy_version
  source_document
  source_pages[]
  confidence
  unresolved_items[]

# phase 3 — independent validation

| validate:

^ for-each result in extract.results:

! independently verify the extracted membership of %result.skill_group%
# re-read the original source material for %group% as a SEPARATE pass — do not reuse or
  re-check the phase-2 intermediate parse; this phase must compare against the source
  itself, not against phase 2's own output
# compare extracted skills against the original source, skill-by-skill
# identify omissions
# identify false inclusions
# identify OCR errors
# identify normalization errors
# verify source pages
$ a check that only confirms internal self-consistency of the phase-2 parse (e.g. "does
  this line start with the expected name") does NOT satisfy this phase and must not be
  reported as independent validation
% return:
  skill_group
  verified_skills[]
  corrections[]
  omissions[]
  additions[]
  confidence

# phase 4 — reconcile

merge extract.results and validate.results

resolve disagreements using this priority:

1. original World Bank/LinkedIn source
2. official LinkedIn publication
3. authoritative secondary reproduction
4. other evidence

If only one primary source was ever located and read (as in the prior run), state this
explicitly rather than implying multi-source reconciliation took place. If the “Skill Group
Definitions” companion source becomes available during this run, reconcile Appendix F
against it using the priority order above and log every contradiction found — do not merge
silently.

never silently resolve contradictory evidence

retain unresolved contradictions in the provenance record

# phase 5 — taxonomy analysis

calculate and report EACH of the following as an explicit named line — including when the
value is zero, “not applicable,” or “not determinable from available sources”:

unique_skill_groups
unique_skills (state explicitly whether this is sample-derived or complete)
skill_group_relationships
skills_in_multiple_groups
unassigned_skills
empty_groups
duplicate_records
unresolved_records

compare unique_skill_groups against any count the source states about itself (e.g. Appendix
F’s own report text says “approximately 250 skill groups”) and report the delta explicitly.

do not force the extracted dataset to match a published count.

# phase 6 — current LinkedIn comparison

| current:

investigate the current LinkedIn Standardized Skills API and current
LinkedIn Skills Graph documentation.

retrieve current skills if API access is available.

keep current data completely separate from the historical dataset.

If API access is not available, state that explicitly in the final report every time this
phase runs — do not omit the phase’s status silently.

compare:

historical_skills
current_skills

identify:

additions
removals
probable renamings
aliases
changed classifications

# phase 7 — publish

Use exactly these filenames — no invented suffixes:

linkedin_skill_groups.csv
linkedin_skill_group_membership.csv
linkedin_skills.csv
linkedin_skill_taxonomy.xlsx
README.md

linkedin_skill_groups.csv

columns:
top_level_category
skill_group
skill_group_definition
taxonomy_version
source_year
source_document
source_page
confidence

(top_level_category and skill_group_definition may legitimately be constant
“not present in source” values given the phase-1 findings, unless phase 1 resolves the
blocked companion source — this is an expected, reportable outcome, not an error, and the
columns must still be present, not dropped.)

linkedin_skill_group_membership.csv

columns:
skill_group
skill_name
taxonomy_version
source_year
source_document
source_page
confidence

(mark clearly, in the README’s normalization-rules section, whether skill_name entries are
sample skills or exhaustive membership for the source in use.)

linkedin_skills.csv

columns:
skill_name
skill_group_count
skill_groups
top_level_categories
taxonomy_versions

linkedin_skill_taxonomy.xlsx

sheets (all seven, each separately populated — Validation and Discrepancies are distinct
sheets, not merged into one):
Groups
Skills
Membership
Categories
Sources
Validation
Discrepancies

README.md

include:
methodology
source inventory
taxonomy versions
extraction rules
normalization rules
validation methodology
discrepancies
limitations

# phase 8 — final quality gate

| quality:

independently verify:

• every skill-group membership has provenance
• every skill group has been processed
• every group has been independently validated — per phase 3’s actual second-pass
  requirement, not merely self-consistency-checked against its own phase-2 parse
• duplicate skills are preserved where legitimately multi-grouped
• historical and current taxonomies are not conflated
• reported counts are reproducible
• unresolved issues are explicitly reported, including at minimum: the status of the
  “Skill Group Definitions” companion source, the sample-vs-complete skill list gap, and
  the “five-category mappings” line below

# phase 9 — final report

% report:

taxonomy versions investigated
authoritative sources (including explicit fetch status for each, per phase 1)
groups discovered
groups successfully extracted
groups independently validated
unique skills recovered (state sample-derived vs. complete)
skill/group relationships recovered
multi-group skills
five-category mappings — this line must be explicitly addressed even if unresolved: state
whether a five-category (or any) broad-category structure was found, in which source, and
if none was found, say so plainly rather than omitting the line
discrepancies
unresolved records
estimated coverage
current-vs-historical differences