the-distro/channel-scripts
1
0
Fork 1
Code Issues 8 Pull requests Packages Projects Releases Wiki Activity
237 commits 1 branch 1 tag 1 MiB
Commit graph

237 commits

This branch
This branch
All branches
Author SHA1 Message Date
Eelco Dolstra 9f96e1e25d
Improve error message 2018-02-06 16:45:13 +01:00
Eelco Dolstra 6a653c367e
Fix build 2018-01-04 15:33:22 +01:00
Eelco Dolstra c3a9ba32ee
Sync with nixUnstable 2017-07-21 12:48:15 +02:00
Eelco Dolstra c6428851dc
mirror-nixos-branch: Update the Git repo last 2017-07-21 12:48:14 +02:00
Eelco Dolstra 2f7994e3de
Remove bad check 2017-07-07 19:42:20 +02:00
Eelco Dolstra b6c049019a
mirror-nixos-branch.pl: Call index-debuginfo 2017-07-07 17:12:25 +02:00
Eelco Dolstra fcd6250fd0
mirror-channel.pl: Remove 
This is no longer used, since Hydra uploads directly to S3 now. Also,
"nix copy" can copy to S3 now.
 2017-07-07 17:06:21 +02:00
Eelco Dolstra a5e125330c
generate-programs-index: Check whether symlink targets exist 
Fixes https://github.com/NixOS/nixpkgs/issues/26661
 2017-07-07 16:45:45 +02:00
Eelco Dolstra 74c917454c
Add a command for indexing debug info files by build ID 
For every file "lib/debug/.build-id/<build-id>.debug" in every store
path ending in -debug in a release, this command creates an S3 key
"debuginfo/<build-id>" specifying the location of the NAR that
contains the debug info file for that build ID. Thus,
https://cache.nixos.org/debuginfo allows debug info files to be looked
up by build ID. This allows debug info files to be fetched
automatically by dwarffs [1].

One minor issue is that in theory, multiple store paths could provide
debug info files for a particular build ID. So then when we garbage
collect a store path from the binary cache, we migth invalidate a
debug info link even though there are non-GC'ed store paths that
provide the file. Ah well.

I could have used HTTP redirects, but they're kind of a pain to
configure with S3.

[1] https://github.com/edolstra/dwarffs
 2017-07-07 15:33:27 +02:00
Eelco Dolstra a91fb566a9
Factor out the file cache 2017-07-06 14:53:52 +02:00
Eelco Dolstra 3daca6b592
Update generate-programs-index to handle .ls 2017-04-18 16:41:11 +02:00
Eelco Dolstra dd0d2d2ca6
Update build 2017-04-18 15:26:38 +02:00
Eelco Dolstra a5368b6239 Merge pull request #12 from primeos/master 
Fix the displayed hash name (SHA-512 -> SHA-256)
 2017-03-31 12:04:05 +02:00
Michael Weiss c3e8e3badb Fix the displayed hash name (SHA-512 -> SHA-256) 
The HTML pages for each channel are showing the SHA-256 hash of each
file but the table header actually referred to it as an "SHA-512 hash".
 2017-03-31 06:39:24 +02:00
Eelco Dolstra 1c07d6dd08
Lock the Git repo 2016-12-05 11:50:17 +01:00
Eelco Dolstra 2fd4d6beea Vacuum programs.sqlite 
This cuts a few hundred KiB from the database.
 2016-11-10 18:27:04 +01:00
Eelco Dolstra 6883275653 Ignore package listings with malformed UTF-8 
For example, https://cache.nixos.org/5q3s4vfwwd9n9499yg11ydziwxgn644m.ls.xz
contains the string "bokm<E5>l.alias". This causes nlohmann::json to
barf.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra 2176c939fc Compute FS closures in parallel 
This is much faster.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra b74e03a870 Revert "Disable programs.sqlite generation" 
This reverts commit 7f2bbc884c.
 2016-11-10 18:10:08 +01:00
Eelco Dolstra ba19471e5b Use the .ls.xz listings generated by Hydra to speed up building programs.sqlite 
This reduces the time to build the index for the regular (non-small) channel from ~10 hours to ~4 minutes.

Fixes #9.
 2016-11-10 18:10:03 +01:00
Eelco Dolstra 6391c4dcec Fix check for release existance 
We no longer have a github-link file, so this check always failed. As
a result, we were constantly re-uploading releases.
 2016-10-21 20:10:53 +02:00
Eelco Dolstra 038c8ee8d4 Update the channels repo early 
This way, users of the nixpkgs-channels repo don't have to wait for
all the slow stuff to finish (like index generation and copying of
ISOs).
 2016-10-21 19:22:05 +02:00
Eelco Dolstra 7f2bbc884c Disable programs.sqlite generation 2016-10-21 18:27:28 +02:00
Eelco Dolstra eea0729481 Generate a simple directory listing 2016-09-29 11:50:14 +02:00
Eelco Dolstra 6cb926311e Only update the release link if the release has changed 2016-09-29 10:55:11 +02:00
Eelco Dolstra fcf5a77736 Add an empty file at the release prefix for nix-channel compatibility 
nix-channel expects the prefix to exist (not return a 404).
 2016-09-26 19:50:36 +02:00
Eelco Dolstra d3d33d5b25 Store releases in S3 
Fixes #7.
 2016-09-26 18:06:51 +02:00
Eelco Dolstra becf1f2ae3 Atomically update channels/.htaccess 
Otherwise we might clobber the redirects if the disk is full, and in
any case there is a time window in which .htaccess is in an
inconsistent state.

Fixes https://github.com/NixOS/nixpkgs/issues/18544.
 2016-09-12 23:20:12 +02:00
Eelco Dolstra 8a0ed20e6d Improve error message 2016-08-22 17:26:27 +02:00
Eelco Dolstra 76ca1da3ee Add $out/bin to $PATH 2016-08-22 17:18:44 +02:00
Eelco Dolstra c981c37438 Add README 2016-08-22 17:18:29 +02:00
Eelco Dolstra ab086b22ef Revive programs.sqlite generation 
Fixes #4.
 2016-08-11 14:41:26 +02:00
Eelco Dolstra 8f7fbdaff7 Die tabs die 2016-08-08 17:32:24 +02:00
Eelco Dolstra 72501d545d Remove obsolete binary patch generation scripts 2016-04-13 11:42:52 +02:00
Eelco Dolstra 852e1b6bf2 Drop 32-bit images (except iso-minimal) 2016-03-17 12:21:13 +01:00
Eelco Dolstra 000a9f2766 Workaround for https://github.com/NixOS/nix/issues/792 2016-03-07 11:12:24 +01:00
Eelco Dolstra 373d8b7b38 Fix NixOS 15.09 channel generation 2016-03-07 10:51:46 +01:00
Eelco Dolstra f46cb14bb7 Handle nixpkgs-unstable channel updates 2016-03-04 20:45:35 +01:00
Eelco Dolstra a6ab71c2f9 Update .htaccess redirects to latest NixOS ISOs 2016-03-04 20:27:15 +01:00
Eelco Dolstra 18af18f351 Fix Git path 2016-03-03 18:29:05 +01:00
Eelco Dolstra a058826231 Record top-level store paths of each release 
We need this to do garbage collection on cache.nixos.org now that
MANIFEST is gone.
 2016-03-03 15:59:25 +01:00
Eelco Dolstra f7beae78eb Rewrite mirror-nixos-branch in perl; handle new Hydra situation 
We're no longer on a machine that contains the actual store paths of
the release, so we have to fetch them.
 2016-03-01 20:03:10 +01:00
Eelco Dolstra 4de7a4a85c Remove tabs 2016-03-01 11:12:36 +01:00
Eelco Dolstra 0c75ea5ba5 Push nixpkgs-unstable branch to nixpkgs-channels 
Fixes https://github.com/NixOS/nixpkgs/issues/11427.
 2015-12-04 18:15:39 +01:00
Eelco Dolstra 5fd290852c Do a git update before doing rev-parse 2015-12-02 11:42:49 +01:00
Eelco Dolstra 97b39b07dc Check for errors when compressing NARs 
Otherwise, an error (like disk full) will cause a truncated NAR file
to be uploaded to the binary cache.
 2015-10-06 23:13:49 +02:00
Eelco Dolstra e4b04c877f Package database: Support multiple outputs 2015-09-02 18:36:17 +02:00
Eelco Dolstra 0cd5fe7808 Prefer shorter attribute paths in the package database 
E.g. "python" is better than "python27Packages.python".
 2015-09-02 18:06:27 +02:00
Eelco Dolstra bc1b2301df Use attribute paths in the package database 2015-09-02 17:39:05 +02:00
Eelco Dolstra d528d56e2d Prevent concurrent access to the Git clone 2015-09-02 11:31:13 +02:00