Arcanist writes cache files to its source directory, and this prevents packaging
Closed, ResolvedPublic
Actions

Asked by fgaz on Oct 3 2023, 14:13.

Details

As written in the title, arcanist mixes user data with its own code, with no way to tell it to put it somewhere else. Specifically, it tries to write libexec/arcanist/src/.phutil_module_cache.

This can be a problem when packaging arc, because the install location is read-only.

Locally, I solved the issue with this patch (apache 2.0 licensed):

diff --git a/src/moduleutils/PhutilLibraryMapBuilder.php b/src/moduleutils/PhutilLibraryMapBuilder.php
index f51fc464..b80a22cd 100644
--- a/src/moduleutils/PhutilLibraryMapBuilder.php
+++ b/src/moduleutils/PhutilLibraryMapBuilder.php
@@ -121,7 +121,10 @@ final class PhutilLibraryMapBuilder extends Phobject {
    * @task path
    */
   private function getPathForSymbolCache() {
-    return $this->getPath('.phutil_module_cache');
+    $cachedir = getenv('XDG_CACHE_HOME');
+    if(!$cachedir)
+      $cachedir = getenv('HOME') . '/.cache';
+    return $cachedir . '/arcanist/phutil_module_cache';
   }
 
   /**
@@ -206,6 +209,10 @@ final class PhutilLibraryMapBuilder extends Phobject {
     }
 
     $json = json_encode($cache);
+
+    if (!file_exists(dirname($cache_file)))
+      mkdir(dirname($cache_file), 0755, true);
+
     Filesystem::writeFile($cache_file, $json);
   }

It works well for me, but cache directories are platform-dependent (example) so the patch is not complete.

Could (a complete version of) this be integrated in the project?

Answer Summary

With D25446, Q77#A106 can be applied

Answers

valerio.bozzolan
Updated 445 Days Ago
Actions

Interesting. Thanks!

Could libexec/arcanist/src/.phutil_module_cache just be a symlink to somewhere else?

avivey
Updated 435 Days Ago
Actions

The location .phutil_module_cache cannot be redirected to $cachedir, because it's always found in the root of each libphutil library - arc is one such library, phorge is another.

But it's only a function of the code itself, so if you're packaging arc as an artifact, you can add the .phutil_module_cache file into the packaged artifact.

You can create the file by running ./support/lib/rebuild-map.php src/ - it takes a few seconds.

See this patch:

D25446: only update cache file if something changed

Event Timeline

The command does create the file, but arc lint tries to modify it anyway:

$ arc lint
 Exception 
Some linters failed:
    - FilesystemException: Requested path '/nix/store/fpz4hr3gnn97nbn3k85y6q62lrahi7cd-arcanist-2023.32/libexec/arcanist/src/.phutil_module_cache' is not writable.
(Run with `--trace` for a full exception trace.)

I'm looking at PhutilLibraryMapBuilder:analyzeLibrary(), and it kinda looks like:

if $this->fileSymbolMap is null, we call analyzeLibrary()`
We load the data from the file if it's there
We update the data
We store it in $this->fileSymbolMap
and we always save it to disk

There's no return in analyzeLibrary`, so if we got there, we'll save to disk. And there's no other write to fileSymbolMap, so we'll always get there.

So a simple change would be to check if there are any updates before saving...

@fgaz can you try this and see if it it works for you?

diff --git src/moduleutils/PhutilLibraryMapBuilder.php src/moduleutils/PhutilLibraryMapBuilder.php
index f51fc464..196ced22 100644
--- src/moduleutils/PhutilLibraryMapBuilder.php
+++ src/moduleutils/PhutilLibraryMapBuilder.php
@@ -472,7 +472,9 @@ EOPHP;
// We're done building the cache, so write it out immediately. Note that
// we've only retained entries for files we found, so this implicitly cleans
// out old cache entries.
-    $this->writeSymbolCache($symbol_map, $source_map);
+    if ($futures) {
+      $this->writeSymbolCache($symbol_map, $source_map);
+    }

// Our map is up to date, so either show it on stdout or write it to disk.
$this->librarySymbolMap = $this->buildLibraryMap($symbol_map);

@avivey Yes this seems to work! Thank you

great - see D25446

New Answer

Answer

This question has been marked as closed, but you can still leave a new answer.

Arcanist writes cache files to its source directory, and this prevents packagingClosed, ResolvedPublicActions

Details

Answer Summary

Event Timeline

Answers

valerio.bozzolanUpdated 445 Days AgoActions

Event Timeline

aviveyUpdated 435 Days AgoActions

Event Timeline

New Answer

Answer

Arcanist writes cache files to its source directory, and this prevents packaging
Closed, ResolvedPublic
Actions

valerio.bozzolan
Updated 445 Days Ago
Actions

avivey
Updated 435 Days Ago
Actions