ALHP
/
ALHP.GO
6
34
Fork 9
Code Issues 29 Pull Requests Projects Releases Wiki Activity

Repository housekeeping #22

New Issue
Open
opened 2021-07-10 21:34:39 +02:00 by anonfunc · 6 comments
anonfunc commented 2021-07-10 21:34:39 +02:00
Owner
Copy Link

Possible things to do:

  • validate .sig files
  • match version on fs to db version
  • check for all split packages (b484f7d4d7)
  • check if package is still part of repo
  • delete packages from db that do not exist anymore
  • remove packages waiting for a dependency over 48 hours (usually means there are some inconsistency with a dependency) (ef61ad7dcf)
  • verify signature in db matches signature in .sig
Possible things to do: - [x] validate .sig files - [x] match version on fs to db version - [x] check for all split packages (https://git.harting.dev/ALHP/ALHP.GO/commit/b484f7d4d7bc743b7ed1a2ab05c63d175869261e) - [x] check if package is still part of repo - [x] delete packages from db that do not exist anymore - [x] remove packages waiting for a dependency over 48 hours (usually means there are some inconsistency with a dependency) (https://git.harting.dev/ALHP/ALHP.GO/commit/ef61ad7dcf40a67580ffeca8f6446fa6e5e729b9) - [ ] verify signature in db matches signature in *.sig*
anonfunc added this to the beta milestone 2021-07-10 21:34:42 +02:00
anonfunc added the
enhancement
 label 2021-07-10 21:34:45 +02:00
anonfunc removed this from the beta milestone 2021-07-10 21:34:49 +02:00
Gontier-Julien commented 2021-11-13 21:50:40 +01:00
Copy Link

Possible things to do:

  • validate .sig files
  • match version on fs to db version
  • check for all split packages (in current versions, ALHP assumes pkgbase X is build if split-package X1 is existing)
  • check if package is still part of repo

I'm on a day to take care of things so let go with this one :D

• to validate .sign, I don't know if this will help

• could you possibly explain more? ^^'

• is this necessary? Unless of a weird bug, I don't see why the X package couldn't be build if the split package is?

• for this, I think download a DB from a fresh trusted repo, then compare it to the appropriate repo, for example:

Refresh to a fresh not out-of-date repo with reflector > compare core-v3.db to core.db > remove packages that are not in core anymore

> Possible things to do: > > * validate .sig files > * match version on fs to db version > * check for all split packages (in current versions, ALHP assumes pkgbase X is build if split-package X1 is existing) > * check if package is still part of repo I'm on a day to take care of things so let go with this one :D • to validate .sign, I don't know if [this will help](https://wiki.archlinux.org/title/GnuPG#Verify_a_signature) • could you possibly explain more? ^^' • is this necessary? Unless of a weird bug, I don't see why the X package couldn't be build if the split package is? • for this, I think download a DB from a fresh trusted repo, then compare it to the appropriate repo, for example: Refresh to a fresh not out-of-date repo with reflector > compare core-v3.db to core.db > remove packages that are not in core anymore
anonfunc commented 2021-11-13 22:03:54 +01:00
Author
Owner
Copy Link

I actually already implemented some of these things (see de3ca80aab/utils.go (L491) ). Just not all. Therefore it's not active yet (and it needs some testing beforehand, obviously).

Possible things to do:

  • validate .sig files
  • match version on fs to db version
  • check for all split packages (in current versions, ALHP assumes pkgbase X is build if split-package X1 is existing)
  • check if package is still part of repo

I'm on a day to take care of things so let go with this one :D

• to validate .sign, I don't know if this will help

Already done.

• could you possibly explain more? ^^'

Just a consistency check if the database still represents what is really in the repo.

• is this necessary? Unless of a weird bug, I don't see why the X package couldn't be build if the split package is?

Well, the whole housekeeping is a "should not happen" kind of thing, but that's why it's being checked.

• for this, I think download a DB from a fresh trusted repo, then compare it to the appropriate repo, for example:

Refresh to a fresh not out-of-date repo with reflector > compare core-v3.db to core.db > remove packages that are not in core anymore

I plan to solve that by querying the mirror for that package and check for the repo listed there. If they match: nice, if not: remove package from repo.

I actually already implemented some of these things (see https://git.harting.dev/anonfunc/ALHP.GO/src/commit/de3ca80aab8d1d85bbc0644d48cbc2214cb52622/utils.go#L491 ). Just not all. Therefore it's not active yet (and it needs some testing beforehand, obviously). > > Possible things to do: > > > > * validate .sig files > > * match version on fs to db version > > * check for all split packages (in current versions, ALHP assumes pkgbase X is build if split-package X1 is existing) > > * check if package is still part of repo > > I'm on a day to take care of things so let go with this one :D > > • to validate .sign, I don't know if [this will help](https://wiki.archlinux.org/title/GnuPG#Verify_a_signature) > Already done. > • could you possibly explain more? ^^' > Just a consistency check if the database still represents what is really in the repo. > • is this necessary? Unless of a weird bug, I don't see why the X package couldn't be build if the split package is? > Well, the whole housekeeping is a "should not happen" kind of thing, but that's why it's being checked. > • for this, I think download a DB from a fresh trusted repo, then compare it to the appropriate repo, for example: > > Refresh to a fresh not out-of-date repo with reflector > compare core-v3.db to core.db > remove packages that are not in core anymore I plan to solve that by querying the mirror for that package and check for the repo listed there. If they match: nice, if not: remove package from repo.
Gontier-Julien commented 2021-11-13 22:07:27 +01:00
Copy Link

I actually already implemented some of these things (see de3ca80aab/utils.go (L491) ). Just not all. Therefore it's not active yet (and it needs some testing beforehand, obviously).

Possible things to do:

  • validate .sig files
  • match version on fs to db version
  • check for all split packages (in current versions, ALHP assumes pkgbase X is build if split-package X1 is existing)
  • check if package is still part of repo

I'm on a day to take care of things so let go with this one :D

• to validate .sign, I don't know if this will help

Already done.

• could you possibly explain more? ^^'

Just a consistency check if the database still represents what is really in the repo.

• is this necessary? Unless of a weird bug, I don't see why the X package couldn't be build if the split package is?

Well, the whole housekeeping is a "should not happen" kind of thing, but that's why it's being checked.

• for this, I think download a DB from a fresh trusted repo, then compare it to the appropriate repo, for example:

Refresh to a fresh not out-of-date repo with reflector > compare core-v3.db to core.db > remove packages that are not in core anymore

I plan to solve that by querying the mirror for that package and check for the repo listed there. If they match: nice, if not: remove package from repo.

Okay awesome ^^

> I actually already implemented some of these things (see https://git.harting.dev/anonfunc/ALHP.GO/src/commit/de3ca80aab8d1d85bbc0644d48cbc2214cb52622/utils.go#L491 ). Just not all. Therefore it's not active yet (and it needs some testing beforehand, obviously). > > > > Possible things to do: > > > > > > * validate .sig files > > > * match version on fs to db version > > > * check for all split packages (in current versions, ALHP assumes pkgbase X is build if split-package X1 is existing) > > > * check if package is still part of repo > > > > I'm on a day to take care of things so let go with this one :D > > > > • to validate .sign, I don't know if [this will help](https://wiki.archlinux.org/title/GnuPG#Verify_a_signature) > > > > Already done. > > > • could you possibly explain more? ^^' > > > > Just a consistency check if the database still represents what is really in the repo. > > > • is this necessary? Unless of a weird bug, I don't see why the X package couldn't be build if the split package is? > > > > Well, the whole housekeeping is a "should not happen" kind of thing, but that's why it's being checked. > > > • for this, I think download a DB from a fresh trusted repo, then compare it to the appropriate repo, for example: > > > > Refresh to a fresh not out-of-date repo with reflector > compare core-v3.db to core.db > remove packages that are not in core anymore > > I plan to solve that by querying the mirror for that package and check for the repo listed there. If they match: nice, if not: remove package from repo. Okay awesome ^^
👍 1
anonfunc commented 2021-11-19 23:05:42 +01:00
Author
Owner
Copy Link

Housekeeping is now enabled. It already found some issues, so far working as expected.

Housekeeping is now enabled. It already found some issues, so far working as expected.
🎉 2
Gontier-Julien commented 2021-11-19 23:08:45 +01:00
Copy Link

Housekeeping is now enabled. It already found some issues, so far working as expected.

Nice that amazing! Great work anonfunc !!

> Housekeeping is now enabled. It already found some issues, so far working as expected. Nice that amazing! Great work anonfunc !!
👍 1
anonfunc commented 2021-11-20 01:19:52 +01:00
Author
Owner
Copy Link

There were around 100 orphaned packages removed, in addition about half as much moved repos (and are going to be rebuild).

If you do notice anything strange in the next few days (like dependency problems or packages not being listed on the package status even if they are available in your mirror/archweb), please comment here.

There were around 100 orphaned packages removed, in addition about half as much moved repos (and are going to be rebuild). If you do notice anything strange in the next few days (like dependency problems or packages not being listed on the package status even if they are available in your mirror/archweb), please comment here.
🚀 1
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
main
Labels
Clear labels   
blocked upstream

   
bug

Something is not working

  
build-failure

package build fails because of our added buildflags

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
informational

Provides information about something

  
invalid

Something is wrong

  
invalid/corrupt package

Package needs to be rebuild

  
packaging issue

   
priority: high

High priority

  
question

More information is needed

  
support

   
wontfix

This won't be fixed

No Label
blocked upstream
bug
build-failure
duplicate
enhancement
help wanted
informational
invalid
invalid/corrupt package
packaging issue
priority: high
question
support
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: ALHP/ALHP.GO#22
No description provided.
Delete Branch "%!s(<nil>)"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?