Ensure that bulk CVE ingestion is done at first deployment #368

New issue

Open

opened 2024-11-18 20:26:47 +00:00 by fricklerhandwerk · 2 comments

fricklerhandwerk commented

2024-11-18 20:26:47 +00:00

(Migrated from github.com)

Right now operators need to remember to run manage ingest_bulk_cve when setting up the service. This is easy to forget, and there's currently no monitoring for startup errors, so everyone will wonder 24h later why there's no data in the suggestion queue.

This can be solved in multiple ways:

run the command in PreStart and leave a flag in the database to ensure the command is idempotent
document it in the operator's manual in a visible place

Right now operators need to remember to run `manage ingest_bulk_cve` when setting up the service. This is easy to forget, and there's currently no monitoring for startup errors, so everyone will wonder 24h later why there's no data in the suggestion queue. This can be solved in multiple ways: - run the command in `PreStart` and leave a flag in the database to ensure the command is idempotent - document it in the operator's manual in a visible place

Erethon commented

2025-04-29 09:44:00 +00:00

(Migrated from github.com)

Is this still valid? Aren't ingestions handled automatically via the worker?

fricklerhandwerk commented

2025-04-29 11:44:26 +00:00

(Migrated from github.com)

Aren't ingestions handled automatically via the worker?

Yes, but it's neither done on startup (but at 03:00 UTC) nor is it documented how to trigger it or that it needs triggering. The problem is therefore that a fresh deployment will simply not do anything useful and there's no indication what to do about it. And as we discussed, reading the code to figure it out is not a particularly pleasant onboarding experience.

> Aren't ingestions handled automatically via the worker? Yes, but it's neither done on startup (but at 03:00 UTC) nor is it documented how to trigger it or that it needs triggering. The problem is therefore that a fresh deployment will simply not do anything useful and there's no indication what to do about it. And as we discussed, reading the code to figure it out is not a particularly pleasant onboarding experience.