Career Timeline — Hyunwook Kim at Labradorlabs (2021–2025)

# Career Timeline — Hyunwook Kim at Labradorlabs (2021–2025)

AI Summary

Purpose:

• Year-by-year record of 김현욱 (Hyunwook Kim)'s work at Labradorlabs from 2021 to 2025, including main projects, role evolution toward data-part lead, and key achievements.

Key points:

• 2021: Built the foundation of the library/license data-collection crawlers; unified the crawler stack on Python; designed language-split DB schema.
• 2022: Expanded into OS package vulnerability collection; built the first version of LPP (Labrador Patch Priority) scoring; added component popularity scoring and differential crawling.
• 2023: Re-architected the file/function vulnerability collection system (V4); solved the hash-gen function-abstraction issue; built the license-compatibility DB.
• 2024: Led overall data-infrastructure architecture (hybrid IDC/in-house/AWS); split the distribution DB into instances; stabilized Synchronizer and designed the binlog shipping service (internal BTS) with binlog scrambling; ran MySQL 8.0 → 8.4 upgrade with benchmarking.
• 2025: Re-engineered the lab-master DB and index set (2.6TB → 1.2TB); built AI-based license collection (V3, ChatGPT API); planned binlog shipping service v2.0.0 (sync-agent log collector, terminology/API versioning).
• Role evolved from crawler/DB engineer to data-part lead and infrastructure architect.

Relevant when:

• An agent needs the chronological context of the author's projects, or needs to map a project (binlog shipping/internal BTS, LPP, crawlers, license, DB optimization) to a year and role.

Do not read full document unless:

• You need the specific per-project background, actions, and outcomes for a given year.

Linked documents:

• [[hyunwook.md]]
• [[../concepts/mysql-binlog.md]]
• [[../concepts/troubleshooting-playbook.md]]
• [[../../sources/vault/os-pkg-binary-collection.md]]

Open Questions

• Exact job titles per year and the precise date of promotion to data-part lead are not stated in the source notes. Marked as "Needs confirmation".
• Some metrics (e.g., "12 OS distributions", "8 programming languages") come from self-authored summary notes; treat as author-reported.

Details

Source: self-authored yearly work summaries (Obsidian vault, 100. 개인/01. TechNotes/02. 래브라도랩스). All facts below are as stated by the author.

Public portfolio wording notes:

• Public pages should generalize internal-only labels. Avoid Jira/Confluence, BTS, internal team names, "part leader", and unverifiable "domestically first" claims.
• Use public labels such as "issue management", "document management", "binary-log-based data synchronization", "LabradorLabs", "Data / Systems Engineer", and "Data Engineering Lead".
• Use capability-based project titles instead of internal version labels when possible: pipeline redesign, monitoring, encryption transition, DB instance separation, common execution package.

2021 — Foundation & Expansion

Theme: building the core data-collection infrastructure of the Labrador DB from the ground up.

• Programming-language library crawlers: developed new crawlers for Go, Swift, .NET (analyzing pkg.go.dev, CocoaPods, NuGet). The Go crawler used a hybrid web-crawl + Git-command approach to maximize collection rate. Re-implemented existing Java-based crawlers — JavaScript (NPM), Python (PyPI), PHP (Packagist), Ruby (RubyGems) — in Python, unifying the crawler stack.

- Outcome: library-info collection across 8 major programming languages; crawler stack unified on Python for maintainability.

• Database schema design: split the unified TB_VERSION table into per-language tables (TB_VERSION_RUBY, TB_VERSION_PHP, ...). Moved license pattern-mapping from text files into the MAPPING column (JSON) of TB_LICENSE_SPDX, enabling regex-based dynamic license matching.
• Performance research: benchmarked REPLACE INTO vs UPDATE vs INSERT ... ON DUPLICATE KEY UPDATE; standardized on INSERT ... ON DUPLICATE KEY UPDATE as the most stable/efficient write pattern.

2022 — Refinement & Value Creation

Theme: extend collection into high-value OS package vulnerability data and add analytical value (LPP).

• OS package vulnerability collection upgrade: analyzed security-data delivery (API, OVAL, Security Tracker) across 12 major OS distributions (Red Hat, Debian, Ubuntu, ...). Reworked crawlers to capture accurate PATCHED_VERSION data. For Ubuntu, switched from web parsing to cloning the Git repository directly, raising data reliability.
• LPP (Labrador Patch Priority) v1: designed a multi-dimensional vulnerability-prioritization model beyond CVSS — using PoC existence, CWE Top 25 membership, attack complexity (AC), required privileges (PR), and CIA impact. Built metadata crawlers, designed the LPP scoring algorithm, and defined the TB_VULN_LPP_SCOREBOARD schema. Described by the author as a domestically-first multi-dimensional patch-priority system; public portfolio copy should avoid this claim unless externally verified and instead describe the multi-dimensional model.
• Component detail collection: used GitHub/GitLab GraphQL APIs to collect Stars, Forks, Commits, contributor counts. Built a popularity/importance scoring model and a differential collection strategy (higher-scored components updated more frequently) to use crawling resources efficiently.

2023 — Deep Tech & Refactoring

Theme: pay down technical debt and reach higher data accuracy by re-designing core modules.

• File/function vulnerability collection V4: split the single-process V3 (crawler-vuln-func) into two stages — crawler-vuln-commitURL (collect/verify patch commit URLs) and crawler-vuln-source (collect source/diff from commit URLs). Redesigned/normalized the V4 schema; added a processed state flag on TB_VULN_COMMIT_URL for trackable, retryable processing.
• hash-gen function-abstraction fix: root-caused inconsistent function-signature generation to (a) the cTag parser mishandling keywords like final, Map<>, and (b) flaws in hash-gen's internal regex. Solved JavaScript anonymous-function naming by generating a content-based hash to produce stable function names.
• License-compatibility DB: built a crawler for osadl.org's license-compatibility matrix; designed TB_LICENSE_COMPATIBILITY to query compatibility (Yes / No / Unknown) between two licenses.

2024 — Infrastructure & Scalability

Theme: secure stability and scalability of the data-distribution system; led overall infra architecture.

• Overall data-infrastructure architecture: designed a hybrid cloud combining IDC (main distribution), in-house (internal dev + backup), and AWS (commercial service), to reduce AWS cost. Designed the end-to-end data flow: collection DB (GATH) → distribution DB (DIST) → development DB (DEV) → customer on-premise. Configured Master–Replica replication between IDC and in-house; wrote scheduled DB auto-backup scripts for disaster recovery.
• Distribution DB instance separation: split DIST into dist-filecomp (file-component data only) and dist-other (everything else) so the large, frequently-updated file-component sync no longer blocked freshness of library/vulnerability data. Each instance produces its own binlog; the customer-side binlog shipping service pulls from both in parallel.
• Data-distribution pipeline — Synchronizer + binlog shipping service (internal BTS): stabilized the Synchronizer (formerly "Copier") that replicates GATH → DIST (error tracking, stronger retry). Strengthened binlog shipping security with binlog scrambling — XOR-scrambling part of the binlog header so transferred content is not trivially readable. Performed recurring incident analysis on DIST-M/R and devDB (disk I/O bottleneck, network disconnect, OOM) via syslog and MySQL error logs.
• MySQL 8.0.x → 8.4.x upgrade: pre-benchmarked with sysbench (up to ~3B rows) and root-caused an HDD write-performance regression on 8.4.x to the changed innodb_flush_method default (fsync → O_DIRECT). Tuned per-hardware (HDD/SSD) and upgraded DEV, DIST-A, DIST-M/R to 8.4.0; established an innodb_flush_method guideline.

2025 — Re-architecture & Innovation

Theme: clear legacy debt, prepare for scale, adopt AI.

• DB re-engineering and index optimization: analyzed 95 real cdbvdb_api queries with EXPLAIN and showed 224 of 311 indexes (72%) on lab-master were unnecessary. Removed them and reordered composite-index columns. Applied SQL tuning (OR → UNION, optimizing LIKE wildcard usage for index use). Outcome: DB size 2.6TB → 1.2TB (−53.8%); initial-install DB creation time 12h → 6h.
• AI-based license collection (V3): simplified the previous multi-table structure (TB_LICENSE_SPDX, TB_LICENSE_DEJACODE, TB_LICENSE_OLIS). Store raw license full_text in TB_LICENSE_RAW; use the ChatGPT API to classify type (permissive/protective) and key permissions into TB_LICENSE_CONDITION_AI, producing a single refined TB_LICENSE_V2. Removed the external dejacode dependency. Further subdivided protective into Weak/Strong Protective via prompt engineering.
• Binlog shipping service v2.0.0 planning: designed a sync-agent log collector so customer-side errors are reported to a central monitoring server for proactive incident response. Standardized data-type terminology (other → main, filecomp → source) and introduced API versioning (v2 → v3) for maintainability.

Role evolution

• Started as crawler/DB engineer building data-collection systems (2021–2023).
• Moved into infrastructure architecture and led concrete execution tasks (DIST instance separation, Synchronizer/binlog shipping service) in 2024.
• By 2024–2025, operating as the data-part lead and infrastructure architect, owning the end-to-end data pipeline blueprint and high-availability/disaster-recovery design. Exact title/date: Needs confirmation.