LLM WikiAccess-protected knowledge portal

WIKI

2026-W23 Worklog

AI Summary Purpose Tracks development work across repositories for this week. Key points Add coding work entries here after meaningful agent sessions. Keep entries concise. Promote durable knowledge to ai/repo notes/ or ai/wiki/ . Relevant

경로ai/worklog/2026/2026-W23.md
카테고리Worklog
태그#ai-review #crawler #infra #license #report #w23 #worklog

# 2026-W23 Worklog

AI Summary

Purpose:

Key points:

Relevant when:

Do not read full document unless:

Linked documents:

Work Items

Dotnet crawler: NuGet catalog missing-data fixes

flow (index -> page -> leaf) against the crawler implementation.

- PACKAGE_UPDATE_URL had a leading whitespace, which can prevent requests.get from reaching NuGet catalog index. - updatePackageListCrawler returned early when TB_COMP_LIB_DOTNET_LIST was empty, so a fresh/truncated LIST could never seed catalog page URLs. - dataCrawling continued past failed catalog leaf fetches without setting error_flag; the catalog page could then be marked PROCESSED=1, leaving failed leaf versions omitted. - early page-fetch failure branches appended status in memory but could skip the insertPackageList call when the failed page was last in the batch. - product SOURCE_URL was read from page item metadata; NuGet projectUrl lives on the catalog leaf.

- src/ai/labradorlabs/input/crawlerInfo.py - src/ai/labradorlabs/crawler/dotnetCrawler.py - Added regression tests in tests/test_dotnet_missing_regressions.py. - Added tests/dependency_stubs.py so host-side tests can run without the Docker image's third-party packages installed.

- python3 -m unittest discover -s tests -v -> 15 OK. - python3 -m compileall src tests -> OK. - git diff --check -> OK.

ai/workspace/repos.md and ai/repo-notes/index.md.

intentionally disabled since commit e0b8086; retry semantics still use PROCESSED=0 rather than a retry ladder; no live NuGet/DB integration test was run in this session.

are case-insensitive, so NuGet PRODUCT_KEY identity should not split casing-only differences. Changed the normal scheduler from daily midnight to every 3 hours (SCHEDULE_INTERVAL_HOURS=3, schedule.every(3).hours) and added tests/test_main_schedule.py. Verified: python3 -m unittest discover -s tests -v -> 17 OK; python3 -m compileall src tests -> OK; git diff --check -> OK.

only when NuGet API leaf metadata has no license value; NuGet-provided license values no longer trigger GitHub requests. Removed libraries.io fallback code and added regression coverage. Verified: python3 -m unittest discover -s tests -v -> 20 OK; python3 -m compileall src tests -> OK; git diff --check -> OK.

page-cached. When NuGet has no license and projectUrl is GitHub, the crawler calls GET /repos/{owner}/{repo}/license once per catalog page/repo and shares that default-branch repo license across same-page versions from the same repo. This intentionally avoids tag-specific API calls to reduce rate-limit pressure; clone/licensee remains fallback only when the repo license API has no data. The API response feeds version license matching and LICENSE_AI, and records ORIGINAL_DATA.GITHUB.LICENSE_API.ref as None. Verified: python3 -m unittest discover -s tests -v -> 26 OK; python3 -m compileall src tests -> OK; git diff --check -> OK.

Go crawler V2 worker: process newest events first

incremental worker claims the newest-timestamp events first (src/ai/labradorlabs/crawler/goCrawler.py).

crawlerInfo.LIMIT_SELECT_QUERY (... LIMIT :start, 100), NOT the BaseCrawler.LIMIT=1000 constant (that is a separate full-select pagination loop, not the worker queue fetch).

(run resync path, line ~66) keeps TIMESTAMP ASC since a full historical backfill should walk oldest->newest.

lock_skip_offset resets to 0, so the next fetch re-queries offset 0 = freshest unprocessed event; lock_skip_offset only advances to skip past rows locked by other workers. attempted_ids still guards in-run repeats.

The V2 integration test passes its own order_by arg to _claimV2Batch, so the _runV2 default flip does not affect it.

Go crawler V2 worker: batch the existence pre-skip

100 single-row getDBVersionExists SELECTs per batch — which was the slow part.

- New _batchSkipExistingEvents(events): for skip-check-eligible events (skip mode on, not resync), collect all (index_path, index_version) pairs and run one row-value IN SELECT (_existingVersionKeys) against the version table; events already present are completed with processed=1 in a single insertPackageListV2 upsert; only the missing ones are returned for scraping. - dataCrawlingV2 now calls it once up front and the per-event loop no longer does the step-1 SELECT (step-2 go.mod resolve + step-3 declared skip + step-4/5 store unchanged). - Collation parity preserved: _normVersionKey uses exact match when targeting gatheringdb (as_cs, the prod GO_TARGET_GATHERINGDB=1 path) and casefold for legacy labradordb (ai_ci) — mirrors the old per-event getDBVersionExists semantics. Minor caveat: ai_ci is also accent-insensitive; casefold only folds case (rare for Go paths).

-> 46 OK; ast.parse OK.

needs a live MySQL 8 + the docker image. The sandbox could not pull the mysql:8.0 base image (restricted registry), so the dockerized run could not complete. The test_resync_column_controls_db_check case is the one that exercises this path and should be run before deploy. Run on a host with registry access: ``sh docker network create gocrawler-test-net docker run -d --name gocrawler-mysql --network gocrawler-test-net \ -e MYSQL_ROOT_PASSWORD=testpw -e MYSQL_DATABASE=gatheringdb mysql:8.0 docker run --rm --network gocrawler-test-net \ -v "$PWD/src:/workspace/src" -v "$PWD/tests:/workspace/tests" \ -e PYTHONPATH=/workspace/src \ -e TEST_DB_URL="mysql+pymysql://root:testpw@gocrawler-mysql:3306/gatheringdb" \ -e GO_LIST_SCHEMA=v2 -w /workspace iotcube/golangcrawler:0.1.0 \ python3 -m unittest tests.test_go_crawler_v2_integration -v ``

Go crawler: batch size 100 -> 300 + compose log volume

Updated the asserting test (test_limit_select_query_claims_100... -> ...300...). Affects all getLimitSelectsDB callers (V2 worker, V1 incremental, resync).

/data/logs/crawler-lib-golang:/workspace/log on all 3 services (main, worker, backfill).

3 parallel workers — keeps the per-claim GET_LOCK loop (run under the global QUEUE_CLAIM_LOCK in _claimV2Batch) short, bounds crash blast radius (events stuck in processing/20 are NOT reclaimed by the retryable filter processed IN (0,30..33)), and keeps load distribution even.

- one host: docker compose up -d main then docker compose up -d --scale worker=3 worker - multi-host: docker compose up -d main on main server, docker compose up -d worker on each of 3 worker servers.

Go crawler: product LATEST_VERSION ordering + version-sort +incompatible fix

before the per-event loop. product is 1 row/module via ON DUPLICATE KEY UPDATE (last-write-wins), so under newest-first (DESC) claiming the newest version must be written LAST -> LATEST_VERSION stays newest. New integration test test_product_latest_version_is_newest_under_desc_claim (+ _seed_event ts arg, _claim_and_crawl order_by arg).

before parsing. Before, v2.0.0+incompatible failed _GO_VERSION_RE and sorted to the bottom as invalid. Regression test test_sort_go_versions_ignores_build_metadata_for_incompatible.

clean run; the NEW product-latest case was NOT re-run locally because the docker daemon kept wedging this session (env issue, not code). Run it on a stable-docker host (see repo-notes command) for 10/10.

sees one version per event); needs per-row sort key or per-module recompute. Deferred per user (this round was the +incompatible compare fix only).

Portfolio: standardize OS package vulnerability terminology

vulnerability" / generic "OS vulnerability" to "OS package vulnerability" in human/portfolio/index.html and human/portfolio/items/os-vuln-collection-accuracy.html.

ai/wiki/people/career-timeline.md and ai/wiki/people/hyunwook.md to use "OS package vulnerability collection/data" while preserving the original meaning of 12 OS distributions/families.

visible portfolio terms for on-premise, 바이너리 로그/Binary Log, Updater / Download Proxy / Importer, 파일/함수 취약점, 오탐·미탐, and API 버저닝. Left Kubernetes/K8s, In-house/사내, and role/team wording context-specific by user-approved recommendation.

human/portfolio, renamed public BTS wording and slugs to "바이너리 로그 쉬핑 서비스", removed "국내 최초" from the patch-priority portfolio copy, and added the customer-security constraint behind the binlog shipping system (external DB replica connection was not acceptable for some customer environments). Added an AI-facing wording note so future public outputs use generic issue/document-management terms and binary-log-shipping terminology.

sounding labels in human/portfolio for public readers. Examples: affiliation lines now use only 래브라도랩스(LabradorLabs), internal team/part wording was removed, BTS copy was reframed as 바이너리 로그 기반 데이터 동기화, version-coded project names (V2/V3/V4) were replaced with capability-based titles, and 사내/마스터 wording was changed to 자체 IDC, 운영 환경, primary/replica, or 수집→정제→배포 where appropriate.

by adding hero summary stats, a representative-item spotlight row, category counts in the CSS-only filters, stronger card hierarchy, responsive mobile layout rules, and reordering cards as recent practical work first, then foundational work, then academic/research items.

research-to-practice narrative: Sejong University graduate research in IoT firmware security/fuzzing, followed by LabradorLabs data engineering leadership.

items and promoted ETL 데이터 수집 플로우 재정립 as the first representative item. The spotlight now covers data platform design, object storage, DB optimization, binlog-based synchronization security, K8s/Airflow operations, and on-premise customer delivery.

Rust crate crawler (POC): design spec drafted

ai/workspace/repos.md + ai/repo-notes/index.md.

data format + pipeline, redesigning ONLY the LIST stage for crates.io. Not a full-registry crawl.

of rhwp's 14 tags, resolve a Cargo.lock (cargo generate-lockfile; rhwp has NO committed lock and is a single crate, no [workspace]) and take the transitive external-crate closure; (2) seed — for each user-supplied crate name, register ALL versions from the sparse index (yanked included, flagged).

(index.crates.io) authoritative for existence/versions/deps/yanked; crates.io JSON API enrichment-only.

Rust identity columns are ai_ci + name normalization (Go uses as_cs).

is Rust-specific. New gatheringdb tables TB_COMP_LIB_VERSION_RUST / TB_COMP_LIB_PRODUCT_RUST.

looser + conditional GET; contact User-Agent; 429/Retry-After -> Go-style retry ladder. POC volume ~ few hundred crates -> minutes.

full-registry = conditional GET + daily db-dump diff).

(env vs file), DEPENDENCIES json mapping, rhwp clone access.

Rust crate crawler (POC): scaffolded code + in-repo docs

- crawler/resolver.py (seed mode: name -> sparse index -> all versions; rhwp-lock mode: per-tag cargo generate-lockfile -> parse lock -> external crates), crawler/rustCrawler.py (worker: sparse index authoritative + JSON API enrich -> VERSION+PRODUCT), crawler/crates_client.py (429/Retry-After backoff, contact UA), crawler/rate_limiter.py (1/s API), crawler/rust_utils.py (canonical name fold _->-, sha256 list ID, sparse-index path, dependency-free Cargo.lock parser). - db/rustDBVo.py (LIST upsert/claim/processed, VERSION+PRODUCT one txn), input/crawlerInfo.py (endpoints/rates/UA/tables/14 rhwp tags/seed sources). - sql/ three DDLs, Dockerfile (ubuntu+python3+rustup), docker-compose.yml, seeds.txt, reused db/base.py + util/{config,logger}.py.

unittest tests.test_rust_utils); all src/**.py` byte-compile.

MySQL 8.4.0) + real crates.io. Created the 3 tables there; ran seed quick-xml (rhwp disabled): 102 versions -> 102 LIST rows all PROCESSED=1 (yanked 4/98 flagged) -> 102 VERSION rows (LICENSE MIT, DEPENDENCIES json 13 entries, SCM, RELEASE_DATE) + 1 PRODUCT row (LATEST_VERSION 0.40.1, repo url, COMP_STATUS=1). Local docker MySQL was unusable (daemon wedged on this mac), so verified directly against gatheringdb at the user's instruction. Repo config.ini restored to a no-creds placeholder after the run.

Rust crate crawler (POC): rhwp full collection + underscore-name bug fix

docker MySQL stayed wedged so ran against gatheringdb directly (user-approved).

(14 tags), took the external-crate set -> 180 unique crates.

(not just the lock-pinned version). Changed resolver.resolve_rhwp to gather crate NAMES (+ referencing tags) then enumerate all sparse-index versions, reusing the seed-style enumeration. RHWP_TAGS records referencing tags.

sparse index (crates.io keeps the author's separator: serde_json index is se/rd/serde_json; the -/_ equivalence is publish-uniqueness only). First run lost 20 underscore crates as not_found. Fix: canonical = lowercase only, separator preserved. Updated unit test; 10/10 still pass. Also added a per-crate sparse-index cache in the worker (_index) so the index is fetched once per crate, not once per version.

processed (8878 collected + 283 skip-existing, 0 not_found, 422 yanked), 9161 VERSION rows + 180 PRODUCT rows. Underscore crates verified (serde_json 182, once_cell 67, encoding_rs 62, strum_macros 49). ~20 min wall-clock (bulk = remote DB upserts). Repo config.ini restored to no-creds placeholder.

191 transitive entries), npm packages, fonts, tools. Verified all 31 listed Rust direct deps are present in TB_COMP_LIB_VERSION_RUST (present=31, missing=0) — covered by the 180-crate full collection. npm packages (14 ext: vite/typescript/webpack/puppeteer-core/canvaskit-wasm/... + @rhwp/core,editor) are a DIFFERENT registry; per user, NOT collected (this crawler is crates.io only). Fonts/tools out of scope.

Rust vulnerability collection (BMT): OSV crates.io -> VULN table

MAPPED_TYPE='RE' for RustSec). Source = OSV crates.io dump (~2479 advisories; aggregates RustSec + GHSA). VULN_ID = CVE alias > RUSTSEC > OSV id. VULN_RANGE converted from OSV SEMVER events to the PyPI-table interval notation ((,X), [a,b), a], | union); introduced 0/0.0.0/0.0.0-0 = unbounded lower (the 0.0.0-0 sentinel was the one fix after first run).

(download/parse/filter/upsert), db/rustDBVo.upsert_vuln_rows + collected_crate_names, app/main_vuln.py, sql/TB_COMP_LIB_VULN_CARGO_V1.sql (renamed from ..._RUST_V1 on 2026-06-03), crawlerInfo VULN_* config. Tests total 23/23 pass; all src compiles.

rhwp crates (smallvec 5, tar 5, futures-* / image / bumpalo / shlex multiple). scope=all would load the whole crates.io advisory set. Repo config restored to no-creds placeholder.

crates.io (used here), GitHub Advisory (RUST ecosystem). NVD/CVE direct mapping is weak (no crate CPE) — follow via OSV/RustSec aliases.

in-repo: README.md, docs/schema.md (the three tables + exactly how each field is stored), docs/ONBOARDING.md (5-min summary + flow + code map + run/verify). Wiki repo-note keeps the Go lineage as AI design memory.

versions incl. yanked (flagged); rhwp resolves per tag with cargo in image; LANGUAGE=rust/REPOSITORY=Cargo; remote unknown (set at push time).

Rust crawler: license standardization (LICENSE_AI / LICENSE_IDS / STANDARD_LICENSE)

- license_utils.py (pure, 13 tests): SPDX-expression parse (OR/AND//, parens, drop WITH), match against TB_LICENSE_V2 (exact NAME_SHORT/NAME then MAPPING regex) -> STANDARD_LICENSE ([{name,url}] json) + LICENSE_IDS (compact [id,...] json); normalize_ai_license_value Go port. - ai_license.py AiLicenseClient -> in-house AI API (AI_LICENSE_URL, same endpoint as Go config) -> LICENSE_AI (json id list); None if unreachable. - Wired into rustCrawler._build_version_row (per-expr cache) for new collections; licenseBackfill.py + app/main_license_backfill.py for existing rows (STANDARD/IDS from stored LICENSE, no refetch). - rustDBVo: load_license_ref, iter_version_license_rows, update_version_license_fields. crawlerInfo: LICENSE_TABLE, AI_LICENSE_URL.

VERSION rows -> STANDARD_LICENSE/LICENSE_IDS 5151, LICENSE_AI 4999 (AI API answered from python; a few timed out -> NULL). e.g. base64 IDS [247,32] AI [32,247]; byteorder Unlicense OR MIT IDS [394,247]. Repo config.ini restored to no-creds placeholder. AI endpoint kept as overridable non-secret default.

Rust crawler: rename vuln table RUST_V1 -> CARGO_V1

(matches the *_CARGO_* ecosystem naming used by the other vuln tables). RENAME TABLE in gatheringdb (data preserved), new sql/...CARGO_V1.sql (old SQL left as a deprecation pointer — rm blocked in sandbox), crawlerInfo.VULN_TABLE + vulnCollector docstring + wiki updated.

25 rows — exactly the 6 non-CVE ids (RUSTSEC-/GHSA-only: bumpalo RUSTSEC-2022-0078, hashbrown RUSTSEC-2024-0402, paste RUSTSEC-2024-0436, shlex GHSA-286m-6pg9-v42v, tar GHSA-3pv8-6f4r-ffg2, zlib-rs RUSTSEC-2024-0401). A shared-gatheringdb prod job over *_V1 vuln tables appears to keep CVE-format VULN_IDs only (consistent with PyPI V1 being all CVE-...). Implication: non-CVE advisories shouldn't live in *_V1; put them in a RAW table or accept CVE-only in V1. 25 CVE rows remain.

Rust crawler: copy data 165 -> 164 (vuln excluded)

(where it was collected) to 211.115.125.164:53306 / labradordb per user. Connection note: 164 uses port 53306; credential = the 53306-port entry documented in the Go repo config.ini (NOT stored here per the no-secrets rule); 164 has labradordb (no gatheringdb).

was missing on 164), TB_COMP_LIB_VERSION_RUST + TB_COMP_LIB_PRODUCT_RUST (data only; 164 already had matching empty schemas, 28/14 cols).

--skip-triggers --complete-insert --insert-ignore (data-only via --no-create-info for the two existing tables) -> mysql < dump` into 164.

columns (STANDARD_LICENSE/LICENSE_IDS/LICENSE_AI) carried over. 164 already had an empty TB_COMP_LIB_VULN_CARGO_V1 (created 2026-06-03 00:22 by another path); NOT touched — no vuln data moved.

Portfolio: deep-research audit + outcome-led improvements

integrity) + a deep-research workflow (25 web sources fetched, 85 claims, 25 adversarially verified → 9 high-confidence findings; reusable knowledge consolidated in ai/learning/portfolio-best-practices.md).

(hostnames/Jira IDs/customers), 0 title violations (no pre-2026 파트리더), 38 cards ↔ 38 item files ↔ filter counts all consistent.

.links + footer (email [email protected], GitHub hyunwook711 — user-provided, intentionally public); MySQL filter under-tagging fixed (5 cards tagged, count 2→7, verified); "분야별 필터" caption added (pure-CSS filter discoverability); dead .empty element removed; tech_portfolio.png 1254px/1.2MB → 256px/62KB.

for already-verified metrics not surfaced on index cards (no invention — only facts literally present in detail pages). Rewrote 8 card descs: db-index-optimization (311→224/72% unused, 12h→6h), firm-cov-fuzzing (8 devices/150 firmwares, max +237.7%), aws-to-idc-migration (EC2 8대+EFS 과금 중단, 343GB), issue-tracking (완료율 ~85%, 온콜 57건+), sejong-master-thesis (7종 비교, 최대 6.4배), object-storage-seaweedfs (수억~수십억 파일, EC(6+2)), both patents (청구항 10/18개 + dates). Added hero .hero-kpi headline-outcomes line; spotlight subtitle scale fix; differentiated the two near-duplicate Grafana cards (사내 vs 고객사 동기화).

"highlight reel + archive" pattern); 래브라도랩스 filter retained for the planned 개인 category split.

scope facts from detail pages onto 4 more Class-C cards — onprem-binlog-shipping-support (보안 제약별 5가지 구성: 기본·파일 검수·시간 제한·폐쇄망·중계 경로), deployment-db-policy (역할·권한 5개 분리 + 6단계 변경 흐름), malicious-package-analysis (3대 유형 + npm·PyPI·Maven·Go 4개 생태계), component-popularity-scoring (GitHub·GitLab GraphQL, Stars·Forks·Commits·기여자 지표). Quantified cards: 5 → 18 of 38.

crawler-modernization, library-table-redesign, vulnerability-verification, ai-license-pipeline-v3, db-backup-automation, binlog-scrambling, db-instance-separation, function-abstraction-hashgen, patch-priority-system, binlog-shipping-encryption, etl-flow-redesign, grafana-monitoring) have no unsurfaced quantitative facts in any source — quantifying them further requires user-provided metrics (asked; per no-invention rule left as-is).

래브라도랩스 work items were performed solo (기획·설계·구현·운영, 기여도 100%; academic 5 are co-authored, untouched). Added a solo-execution line to the hero + intro of index.html and made 단독 수행 explicit in the 역할 section of all 33 item pages (one sentence-level edit each; role titles preserved — no 파트리더; honest phrasing kept where the surrounding process involved the team, e.g. onprem 1차 지원, team-doc rollout). Verified: 단독 absent only from the 5 academic files, count 1 per work file, 파트리더 0, HTML balance clean across all 39 files.

Atlassian MCP from "[ENHC|OBJ] 오브젝트 스토리지 구축 방안" (page 4105765126) and "[ISSU|DBPT] 기존 ETL 데이터 수집 플로우 문제점" (page 4033708033) plus user-stated figures (total data 30~60억 건; legacy recollection = 1,000만~1억 외부 요청/회, rate-limit bound). - etl-flow-redesign: rewrote 배경/접근/임팩트 with the one-take-collection problem → RAW-preserve/re-parse architecture (재처리 외부 요청 0건, 설계 산정 예 2일→5분, version_id tracking, Cache Hit/Miss, Collector/Worker split). Card + spotlight subtitle now outcome-led. - object-storage-seaweedfs: scale 30억~60억 건 (메타 수억~십억, JS만 6천만), 2서버 7.3TB×4 = 58TB raw, filer PG 5억~10억 row (TiKV 미채택), XFS 근거 (AG parallelism vs ext4 inode), expanded rejected-alternatives (JuiceFS 3중첩, cron 자체 미러, MinIO inode/Ceph/Garage/cloud egress, Apache 2.0 vs AGPL). Card + hero subtitle updated. - Spotlight: swapped infra-k8s-airflow-ops → aws-to-idc-migration (EC2 8대· EFS 과금 중단, 343GB) per user; ETL kept first per user. Hero KPI line now leads with "30억~60억 건 데이터 플랫폼". No Jira IDs/hostnames leaked (grep-verified); HTML balance clean.

enrichment to the internal tiers — human/reports/2026-{etl-flow-redesign, object-storage-seaweedfs}-onboarding.html and human/slides/2026-{...}- slides.html. Onboarding pages additionally got Confluence source links (internal-only, pages 4105765126 / 4033708033), the Collector/Worker queue mechanics (processed flag, FOR UPDATE SKIP LOCKED), server topology (a~m/n~z split, nginx routing, PG MASTER/REPLICA), and full candidate-comparison detail. HTML balance verified on all 4 files.

- Confluence-sourced (6 items): ai-cicd-review-kit (중앙 킷 + wrapper 5파일, 5 언어 프로파일, 70-component changed-only testing, full-SHA pinning, secret-scan-before-AI-review, false-green 방지 — from [DSGN|CICD] 4082532543, authored by user); crawler-modernization (12개 크롤러 누락 전수 조사, 7종 결함 유형, Java 16.99% 재수집, NPM 3,825,366 중 890 누락/104분 일일 점검); db-instance-separation (filecomp/other 2계열 + master-replica); binlog-shipping-monitoring-v2 (물리 서버 5대, Vector→TimescaleDB→Grafana, N+1 sealed 판정); binlog-shipping-encryption (AES-256-GCM, 빌드 시 키 임베딩, binlog→SQL upsert 변환, 재시도 — 레거시 XOR/바이트 범위 등 약점 디테일은 공개 페이지에서 의도적 제외); db-backup-automation (월 정기, 7단계 절차, 전체 백업 원칙). - User-confirmed (7 items): mysql-8-4 (8.0.29→8.4.0 EOL-driven, 수집1+ 배포2+master, 레플리카 각 2대, 고객사 ~10곳); vulnerability-verification (RHEL 9.4 10,995건 분석 + 이미지 54개 오탐 유형 분류 — 본인 작업 확인); team-doc-governance (태그 0 → 17종); ai-license-pipeline-v3 (SPDX 650건+, 조인 테이블 3개 제거); grafana-monitoring (4개 모니터링 영역 + 크롤러/백업/CI·CD 알림); patch-priority (실탐지 기반 신호); license-compatibility (SPDX 식별자 기준). function-abstraction-hashgen: figures not recalled → kept qualitative per no-invention rule.

quantification to BOTH internal tiers — 13 onboarding pages (human/reports/) and 13 slide decks (human/slides/) covering all 13 items. Verified: HTML balance OK on 26 files, no legacy-weakness details (XOR/byte ranges) added, no customer names in slides (customer counts kept onboarding-only or genericized).

Portfolio: series linking across related items

kickers + .rel cross-link paragraphs at the bottom of detail pages) to 6 series / 19 cards total: 데이터 플랫폼 재구축 ①② (etl-flow-redesign ↔ object-storage-seaweedfs); 바이너리 로그 동기화 ①~④ (scrambling → log-collector → encryption → monitoring-v2); 라이브러리 크롤러 ①~③ (multi-lang → popularity-scoring → modernization); 라이선스 데이터 ①~③ (db-schema-foundations → compatibility-db → ai-pipeline-v3); OS 패키지 취약점 ①② (accuracy → multi-distro revamp); 펌웨어 보안 연구 ①~⑤ (uart → thesis → iotfirmfuzz → iothybridfuzzer → firm-cov). Academic pages use a plain styled <p> (their template has no .rel class). Infra items (AWS→IDC / K8s hybrid / Airflow ops) deliberately NOT serialized — parallel work, not one system's evolution.

Portfolio: series links synced to internal tiers + internal-deploy guide

= 28 files (onboarding .muted series-xlink paragraph in last section; slides muted line on last slide; links are same-directory relative). Slug remaps: binlog-shipping-log-collector -> 2025-bts-monitoring-log- collector, encryption -> 2026-bts-v4-encryption, monitoring-v2 -> 2026-bts-monitoring-v2. 펌웨어 보안 연구 series has NO internal-tier files (academic items were portfolio-only) — skipped. Post-fix: the agent wrote the series-size as the member number ("시리즈 2"); corrected all 28 to the page's own circled numeral via script. Verified: HTML balance + all series hrefs resolve.

Cloudflare Access (email OTP, owner-only) so onboarding/slides can be viewed on the web privately; uses the existing almaajo Zero Trust team.

Portfolio: onboarding hub refreshed to 38 items + series badges

llm-wiki.pages.dev, output dir human): now 38 rows (33 실무 with onboarding+slides links + 5 학술·연구 linking to ../portfolio/items/ since academic items are portfolio-only). Added a 학술·연구 section, series badges (📦🔄🕷📜🛡🔬, 19 total) on member rows, and a series legend in the header. Corrected stale labels carried from the 19-item era: dropped "국내 최초" (patch-priority), "컨테이너 OS" → "OS 패키지 취약점", and pulled the verified figures into titles (MySQL 8.0.29→ 8.4.0, 53.8%, SPDX 650+, AES-256-GCM, RHEL 10,995, etc.). Verified: HTML balance OK, 38 rows, 0 broken links.

Internal site live behind Cloudflare Access + root redirect + favicon

human), protected by Cloudflare Access (Zero Trust team green-pine-5afb, email OTP, owner only). Verified externally: /reports/onboarding-hub, /slides/, /portfolio/ all return 302 -> cloudflareaccess login (was 200 before the Access app existed). Public portfolio at portfolio-equ.pages.dev unaffected.

human/index.html (meta-refresh -> reports/onboarding-hub.html) and human/_redirects (/ -> /reports/onboarding-hub.html 302) as fallback.

reports/favicon.png + slides/favicon.png, and added <link rel=icon> to all 69 onboarding/slide pages + root. Verified HTML balance on 70 files.

blocked in sandbox) — not committed, harmless; delete manually if desired.

Decisions

HISTORY (commit 9789051). User decision: leave as-is (repo is private; same credential lives in the Go repo config.ini). IF this repo is ever made public, rewrite history (git filter-repo) or rotate the credential first.

LIST for crates.io; two LIST sources (rhwp 14-tag resolved-lock transitive + user seed crates); collect ALL versions of each external crate (yanked included). Identity collation flips to ai_ci + normalization. Sparse index = existence, JSON API = enrichment, 1 req/s. Incremental deferred. Spec: ai/repo-notes/crawler-lib-rust.md.

_runV2 default ordering; left resync on ASC.

sorting each batch ascending in dataCrawlingV2 (option A), rather than reverting to ASC or doing per-event max-version SELECTs.

larger V2 SORT_ORDER redesign (per-row key / recompute) to a later round.

main 1 + worker 3 topology. Rejected 1000: with 3 workers the per-claim GET_LOCK loop under the global queue lock serializes claims in the skip-heavy regime, and a hard-killed worker would strand up to ~1000 events in processing (no auto-reclaim). 300 is the round-trip/safety balance; revisit with claim-chunking + processing-zombie reclaim if a larger batch is ever needed.

row-value IN SELECT + 1 bulk processed=1 upsert) instead of per-event getDBVersionExists. Kept the declared-path (step-3) skip per-event since store_path is only known after the go.mod fetch.