lix-community/nix-security-tracker
0
0
Fork 0
Code Issues 94 Pull requests 9 Projects Releases Packages Wiki Activity

Ingestion of evaluation results of any supported channel #8

New issue
Closed
opened 2023-11-05 23:42:04 +00:00 by RaitoBezarius · 7 comments
RaitoBezarius commented 2023-11-05 23:42:04 +00:00 (Migrated from github.com)
Copy link

The security tracker acts on supported channels, and we need to ingest an evaluation of all nixpkgs for any given supported channel at any point in time.

The tracker should subscribe to channel bumps (open problem), see how https://git.qyliss.net/pr-tracker detects them and how https://git.eno.space/label-tracker.git/ tracks them.

Proposal of implementation

Every time a channel bump, repull the repository, extract a worktree of that channel (or git clone via the reference for fast checkout), run nix-eval-jobs on that commit sha and collect the result and archive it as JSON with meta results (!!!).

Run this as a background job or a cron job that can easily be managed by infrastructure people or administrators to perform any maintenance task like cancelling evaluations, restarting evaluations, configuring the number of concurrent evaluations, etc.

Ideas for the future

Expose this data of evaluations publicly and let people access it directly, it's useful in general.

The security tracker acts on supported channels, and we need to ingest an evaluation of all nixpkgs for any given supported channel at any point in time. The tracker should subscribe to channel bumps (open problem), see how https://git.qyliss.net/pr-tracker detects them and how https://git.eno.space/label-tracker.git/ tracks them. ### Proposal of implementation Every time a channel bump, repull the repository, extract a worktree of that channel (or git clone via the reference for fast checkout), run `nix-eval-jobs` on that commit sha and collect the result and archive it as JSON with meta results (!!!). Run this as a background job or a cron job that can easily be managed by infrastructure people or administrators to perform any maintenance task like cancelling evaluations, restarting evaluations, configuring the number of concurrent evaluations, etc. ### Ideas for the future Expose this data of evaluations publicly and let people access it directly, it's useful in general.
RaitoBezarius commented 2023-11-06 00:03:05 +00:00 (Migrated from github.com)
Copy link

I should also be able to use a management command to seed my evaluation results in my database without having to go through nix-eval-jobs through the background task system as a starter.

I should also be able to use a management command to seed my evaluation results in my database without having to go through `nix-eval-jobs` through the background task system as a starter.
Tom-Hubrecht commented 2023-11-17 18:56:45 +00:00 (Migrated from github.com)
Copy link

I believe that to track the channel bumps the easiest way is to regularly fetch https://nixos.org/channels data

I believe that to track the channel bumps the easiest way is to regularly fetch https://nixos.org/channels data
RaitoBezarius commented 2023-11-17 19:17:39 +00:00 (Migrated from github.com)
Copy link

I believe that to track the channel bumps the easiest way is to regularly fetch nixos.org/channels data

It's not certain this API will stay available on the long term, I don't advise to use it.

> I believe that to track the channel bumps the easiest way is to regularly fetch [nixos.org/channels](https://nixos.org/channels) data It's not certain this API will stay available on the long term, I don't advise to use it.
Tom-Hubrecht commented 2023-11-17 19:36:21 +00:00 (Migrated from github.com)
Copy link

Well, the way the other tools do it is by cloning the repo and looking at the ref of the branches linked to the channels

Well, the way the other tools do it is by cloning the repo and looking at the ref of the branches linked to the channels
👍 1
RaitoBezarius commented 2023-11-17 23:16:11 +00:00 (Migrated from github.com)
Copy link

Well, the way the other tools do it is by cloning the repo and looking at the ref of the branches linked to the channels

I think that's the sure way to go, or you can listen to events of the GitHub repository of nixpkgs.

> Well, the way the other tools do it is by cloning the repo and looking at the ref of the branches linked to the channels I think that's the sure way to go, or you can listen to events of the GitHub repository of nixpkgs.
RaitoBezarius commented 2023-12-05 06:55:15 +00:00 (Migrated from github.com)
Copy link

So ingestion was implemented of manually evaluated nixpkgs.

All that's left is, for "perfect":

  • Track channels, e.g. track GitHub repo and poll/get notified about when a channel moves
  • Trigger nix-eval-jobs as a background task for this
  • Reuse the importing entrypoint we have in the manual importer

In the meantime, what we can hack is:

  • Perform evaluation-time configuration of channels to track, i.e. load a fixture
  • Setup crons based on known delays of channels move
  • Pick up channel move manually in systemd timers and run nix-eval-jobs
  • Throw them at the manual importer one by one
So ingestion was implemented of manually evaluated nixpkgs. All that's left is, for "perfect": - Track channels, e.g. track GitHub repo and poll/get notified about when a channel moves - Trigger nix-eval-jobs as a background task for this - Reuse the importing entrypoint we have in the manual importer In the meantime, what we can hack is: - Perform evaluation-time configuration of channels to track, i.e. load a fixture - Setup crons based on known delays of channels move - Pick up channel move manually in systemd timers and run nix-eval-jobs - Throw them at the manual importer one by one
RaitoBezarius commented 2023-12-05 20:50:57 +00:00 (Migrated from github.com)
Copy link

We need to add the meta attributes in the ingester:

  • knownVulnerabilities
  • sourceProvenance
We need to add the meta attributes in the ingester: - knownVulnerabilities - sourceProvenance
👍 2
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
npins-auto-update
No results found.
Labels
Clear labels   
automation

   
backend

something the web server needs to provide the frontend with

  
bug

Something isn't working

  
contributor experience

   
data

something about quality or quantity of ingested data

  
deployment

   
documentation

Improvements or additions to documentation

  
duplicate

This issue or pull request already exists

  
good first issue

Good for newcomers

  
help wanted

Extra attention is needed

  
nice to have

Nice to have, but not necessary for the time being

  
notifications

   
package maintainer

user story for package maintainers

  
performance

   
skin

Anything related to the visual presentation

  
tech debt

Activities that reduce tech debt

  
user story

description or implementation of a workflow

No labels
automation
backend
bug
contributor experience
data
deployment
documentation
duplicate
good first issue
help wanted
nice to have
notifications
package maintainer
performance
skin
tech debt
user story
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: lix-community/nix-security-tracker#8
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?