Talk:Portage SQLite Cache

Should this even be done now?
So, I was on #gentoo last night talking to a fellow who wanted to know why Portage wasn't updating the metadata. After a little back and forth, I decided to consult man emerge. Here's what it had to say, emphasis mine: --metadata

Transfers metadata cache from ${PORTDIR}/metadata/cache/ to /var/cache/edb/dep/ as is normally done on the tail end of an rsync update using emerge --sync. This process populates the cache database that portage uses for pre-parsed lookups of package data. It does not populate cache for the overlays listed in PORTDIR_OVERLAY. In order to generate cache for overlays, use --regen. In versions of portage [sic] >=2.1.5 the --metadata action is totally unnecessary unless the user has enabled FEATURES="metadata-transfer" in make.conf(5)

And man 5 make.conf has this to say about metadata-transfer: metadata-transfer

Automatically perform a metadata transfer when `emerge --sync` is run. In versions of portage [sic] >=2.1.5, this feature is disabled by default. When metadata-transfer is disabled, metadata cache from the ${PORTDIR}/meta‐data/cache/ directory will be used directly (if available) and eclasses in ${PORTDIR}/eclass/ must not be modified except by `emerge --sync` operations since the cache validation mechanism will not recognize eclass modifications. Normally, this issue only pertains to users of the rsync tree since the cvs tree does not contain a metadata/cache/ directory. Users of the rsync tree who want to modify eclasses should use PORTDIR_OVERLAY in order for the cache validation mechanism to work correctly.

Should we put a big fat warning at the top of the page? Should we have it marked as a candidate for deletion? It appears the metadata is generated already, and it's actually a waste of time, overall, to even bother generating it again. --Titan 20:43, 17 September 2009 (GMT)


 * What a load of bullshit. Users who use overlays that modify eclasses (that’s just about everyone who’s not a noob), HAVE to enable “metadata-transfer”. And here it took that process, which usually takes >20 minutes down to ten seconds! So obviously it’s improving speed there. Which is the whole point of this article! It’s only some losers who misunderstood it to mean general speedup. — 88.77.182.179 05:44, 21 May 2012 (GMT)


 * The point of the article is to configure Portage to use SQLite for its caching. If you're happy with the default method, fine, but If you find SQLite to be faster(?) or whatnot and want to use that instead, that's just as fine, so there's nothing wrong with this article really. Ni1s 21:58, 20 September 2009 (GMT)


 * NO, that’s not the point! That’s the solution! The point is, and has always been, that it was a huge annoyance that metadata-transfer took forever for those who had to do it, and sqlite shortened THAT to < ten seconds. Now on your system (see below), it seems that comes at the cost of slower emerge commands. So one has to decide. But calling it all shit is only based on your ignorance of the original point of this all. — 88.77.182.179 05:44, 21 May 2012 (GMT)

Actually it's slower Without SQLite cache           With SQLite cache

time emerge -ptve system real   0m32.783s               1m1.737s user   0m8.100s                0m42.232s sys    0m0.516s                0m10.556s real   0m6.764s                0m7.016s user   0m6.551s                0m6.771s sys    0m0.190s                0m0.224s real   0m6.739s                0m7.009s user   0m6.500s                0m6.750s sys    0m0.227s                0m0.244s

time emerge -ptve world real   1m28.165s               3m18.535s user   1m4.107s                2m21.802s sys    0m8.053s                0m32.086s real   0m34.973s               0m36.030s user   0m34.501s               0m35.512s sys    0m0.427s                0m0.471s real   0m34.844s               0m35.799s user   0m34.350s               0m35.274s sys    0m0.446s                0m0.480s

time emerge -ptve kde-meta real   3m23.047s               4m18.133s user   2m19.320s               2m56.345s sys    0m34.303s               0m47.415s real   0m19.631s               0m20.176s user   0m19.336s               0m19.844s sys    0m0.273s                0m0.295s real   0m19.572s               0m19.930s user   0m19.289s               0m19.599s sys    0m0.267s                0m0.299s


 * Here enabling sqlite had no decisive impact on performance: simple actions became a couple of seconds slower, kde-meta a dozen of seconds faster. Relocating the cache onto tmpfs decreased kde-meta resolution time by ~30%, though (both with and without sqlite). For me not worthy anyway. Orivej Desh 14:08, 30 January 2011 (GMT)

question
hey guys, if i do not add these in /etc/eix-sync.conf:


 * 1) Sync all overlays

@emerge --regen || true
 * 1) Regenerate overlay metadata

eix-sync did not update overlays base, only portage tree. Maybe one of you check this issue and add to wiki