• Configure Diffusion

One thing to note: it's not currently possible to clone via HTTP, likely because of

I believe git-http-backend is usually in /usr/lib/git-core/ so that directory probably just needs to be added to the PATH.

Good catch, thanks!

@Matthew / @deadalnix / @speck — one of you have a second to config that? I'm on mobile or AFK most of today

Edit: resolved

In T15000#211, @chris wrote:

Good catch, thanks!

no problem! Trying to put together a Docker/devcontainer based dev environment to make it easy for potential contributors.

Edit: resolved

Still seem to be getting a 500 error, but no obvious explanation this time around :/

git clone -v https://secure.phorge.it/source/phorge.git
Cloning into 'phorge'...
fatal: unable to access 'https://secure.phorge.it/source/phorge.git/': The requested URL returned error: 500

Fixed for real this time. I had an error in the sudoers file. Thought the webserver was running under a different user. But just cloned rARC with its HTTPS URI successfully, so should be hunky dory now hopefully.

In T15000#219, @chris wrote:

Fixed for real this time. I had an error in the sudoers file. Thought the webserver was running under a different user. But just cloned rARC with its HTTPS URI successfully, so should be hunky dory now hopefully.

Working for me too, thanks!

• move hostname to https://phorge.it

Or did we decide against it? Do we want to keep it secure.?

my vote would be phorge.it - simpler, easier to market, easier to say and easier to link to. - You could auto-redirect various common www., secure. etc.

In T15000#231, @Ekubischta wrote:

my vote would be phorge.it - simpler, easier to market, easier to say and easier to link to. - You could auto-redirect various common www., secure. etc.

There's a vote in https://temp-community-phab.zulipchat.com/#narrow/stream/291520-product/topic/infrastructure.20needs, and we're loosing 🤷‍♀️

Looks like we.phorge.it is the winner coming out of that, with a static site hosted at the apex

Just to keep this here for posterity

Amaury Séchet: So it appeared very clearly over the past few days that putting phorge on the root domain probably not a good idea, because there is a clash of target audience. In phroge itself you want people to be able to see right away what tasks, diffs and so on are relevent to them so they can get on with their work as fast as possible. At the root domain though, this wouldn't make sense. Someone who ends up there and want to learn more about what is phorge, how to install it, why would they want it over, say, github/gitlab, etc... Because this is the entry point, this kinda belong on the root domain, IMO. Nobody wants to contribute before they know what the project is about, so really, it so doesn't make sense to send people on phorge directly, and the people who contribute already know what the project is, so that information is just redundant for them.

I like we.phorge.it because it welcome all contributors. I was reading recently this piece about how gnome is developed: https://blogs.gnome.org/tbernard/2021/06/11/community-power-1/ and that the design team is actually a big part of it, as you'd expect from a project focused on a GUI. In this light, I'm afraid that we are backing of bit of the bias of the crowd that we have here in developer.phorge.it . While not bad, open source has been really bad in general to attract non devs (this has been my experience contributing to many projects over the years). While, certainly, a domain name will not relay drive people away by itself (or if they do, these are probably going the kind of drama queen that we likely don't want in the project anyways), but IMO that goes into the building of the culture that we want here.

Phabricator has been referring to its users as "people". In fact, the application that manage users is called "People". This is a very conscious choice and a choice I'd like to see us uphold. This is why I think "we" is better.

Eric Kubischta: I think that @Amaury Séchet has great points - And I don't have any argument against them. My vote for the domain would still be phorge.it for the primary reason of simplicity. However, I would pre-pend my choice as being contingent on a really good, descriptive and helpful welcome panel that brand-new people would see. My second choice, would be we.phorge.it this is an excellent and marketable domain name. Looks great, sounds great, and easy to remember.

Aviv Eyal: +1 everything Eric just said.

Eric Kubischta: I further want to caution : If this team decides that phorge.it will be some other welcome page / website and xxxx.phorge.it will be Phorge - Then someone or some group has to maintain a second thing....whatever is powering the main website. Since I am not sure how this can be controlled by Phorge policies, I have large concerns over the long-term maintainability of something that is not Phorge that we have to keep up.

Leopold Schabel: Absolutely, but a static website served from a S3 bucket or similar seems quite manageable maintenance-wise. Perhaps we could even (ab)use Phame at first?

Amaury Séchet: @Leopold Schabel yes, that was my thinking. A couple of static pages can go a long way.

Thomas Willson: If it's simple and deployment is triggered by pushes to a Git repo, then Phorge policies ought to be able to manage _most_ changes.

There's some issue with the SSH settings here - probably file permissions?

$ git fetch phorge
From ssh://secure.phorge.it:2222/source/phorge
 * [new branch]            legacy-2019 -> phorge/legacy-2019
 * [new branch]            master      -> phorge/master
 * [new branch]            stable      -> phorge/stable
[2021-06-17 15:38:32] EXCEPTION: (Exception) Unable to write to logfile "/var/log/phorge/ssh.log"! at [<arcanist>/src/filesystem/PhutilDeferredLog.php:193]
arcanist(head=master, ref.master=246e604a070f), phabricator(head=master, ref.master=5e3f4e418f5c)
  #0 <#2> PhutilDeferredLog::__destruct()
  #1 phlog(Exception) called at [<arcanist>/src/filesystem/PhutilDeferredLog.php:201]
  #2 PhutilDeferredLog::write() called at [<arcanist>/src/filesystem/PhutilDeferredLog.php:155]
  #3 PhutilDeferredLog::__destruct()
[2021-06-17 15:38:32] EXCEPTION: (Exception) Unable to write to logfile "/var/log/phorge/ssh.log"! at [<arcanist>/src/filesystem/PhutilDeferredLog.php:193]

Also won't let me push, because something thinks it's a non-fast-forward (it is, unless I'm drunk):

$ git push phorge 2abd75c162:master
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 817 bytes | 408.00 KiB/s, done.
Total 3 (delta 1), reused 0 (delta 0), pack-reused 0
remote: +---------------------------------------------------------------+
remote: |      * * * PUSH REJECTED BY EVIL DRAGON BUREAUCRATS * * *     |
remote: +---------------------------------------------------------------+
remote:              \
remote:               \                    ^    /^
remote:                \                  / \  // \
remote:                 \   |\___/|      /   \//  .\
remote:                  \  /V  V  \__  /    //  | \ \           *----*
remote:                    /     /  \/_/    //   |  \  \          \   |
remote:                    @___@`    \/_   //    |   \   \         \/\ \
remote:                   0/0/|       \/_ //     |    \    \         \  \
remote:               0/0/0/0/|        \///      |     \     \       |  |
remote:            0/0/0/0/0/_|_ /   (  //       |      \     _\     |  /
remote:         0/0/0/0/0/0/`/,_ _ _/  ) ; -.    |    _ _\.-~       /   /
remote:                     ,-}        _      *-.|.-~-.           .~    ~
remote:   *     \__/         `/\      /                 ~-. _ .-~      /
remote:    \____(Oo)            *.   }            {                   /
remote:    (    (..)           .----~-.\        \-`                 .~
remote:    //___\\  \ DENIED!  ///.----..<        \             _ -~
remote:   //     \\                ///-._ _ _ _ _ _ _{^ - - - - ~
remote:
remote:
remote: DANGEROUS CHANGE: The change you're attempting to push updates the branch 'master' from '51cb7a3db9e8' to '2abd75c16237', but this is not a fast-forward. Pushes which rewrite published branch history are dangerous.
remote: Dangerous change protection is enabled for this repository.
remote: Edit the repository configuration before making dangerous changes.
remote:
[2021-06-17 15:41:20] EXCEPTION: (Exception) Unable to write to logfile "/var/log/phorge/ssh.log"! at [<arcanist>/src/filesystem/PhutilDeferredLog.php:193]
arcanist(head=master, ref.master=246e604a070f), phabricator(head=master, ref.master=5e3f4e418f5c)
  #0 <#2> PhutilDeferredLog::__destruct()
  #1 phlog(Exception) called at [<arcanist>/src/filesystem/PhutilDeferredLog.php:201]
  #2 PhutilDeferredLog::write() called at [<arcanist>/src/filesystem/PhutilDeferredLog.php:155]
  #3 PhutilDeferredLog::__destruct()
[2021-06-17 15:41:20] EXCEPTION: (Exception) Unable to write to logfile "/var/log/phorge/ssh.log"! at [<arcanist>/src/filesystem/PhutilDeferredLog.php:193]
To ssh://secure.phorge.it:2222/source/phorge.git
 ! [remote rejected]       2abd75c162 -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://secure.phorge.it:2222/source/phorge.git'

$ git log --oneline 2abd75c162 -2
2abd75c162 (HEAD -> arcpatch-D25000) Update Readme
51cb7a3db9 (phorge/master, origin/master, origin/HEAD, master) Provide an ad-hoc maintenance lock for clustered repositories

In T15000#277, @avivey wrote:

Also won't let me push, because something thinks it's a non-fast-forward (it is, unless I'm drunk):

It could be related to git version on the server? - or locally?

git version 2.32.0.rc3 locally; 2.25.1 on the server. Both reasonably recent...

Same with a patch workflow against a fresh clone of the repo:

phorge  (master)$ arc --config phabricator.uri=https://secure.phorge.it patch D25000
 INFO  Base commit is not in local repository; trying to fetch.
Created and checked out branch arcpatch-D25000.


    This diff is against commit 4042d24d74791f816646e355cf2e4d94cb11391a, but
    the commit is nowhere in the working copy. Try to apply it against the
    current working copy state? (51cb7a3db9e8e510d564f6b37297111f7a690c91)
    [Y/n] y

Checking patch README.md...
Applied patch README.md cleanly.
 COMMITTED  Successfully committed patch.

Was there any weirdness in how you opened the revision or anything? Alternately, does re-applying your patch onto rP51cb7a3db9e8 resolve things?

That one is totally my fault - 4042d24d74 is a local commit I have (updates .arcconfig). But I was trying the push from a different commit, which has 51cb7a3db9 as its (only) parent.

I think there might be some permissions issues with the log location but I'm not sure if it's the root cause of the issue being seen here.

/var/log/phorge/ is user and group writeable by www-data but for VCS operations the VCS user will need to be able to write to those logs as well (I'm pretty sure).

We could make a new group and put both www-data and git into the same group, set that folder/files to be for that group and make it group writeable. I'm not sure if that's the best approach though -- I think newly created files in that folder won't inherit the same group but I'm not sure.

Do we have a documented release strategy? I'm not very familiar with git and I only have a vague sense of what Phabricator's release process was. I think it's something like

• Accepted changes are landed into master
• Evan cherry-picks changes from master into stable to "release"

Possibly with some additional smoke-testing somewhere in all this?

Are cherry-picked commits new commits with new hashes or is it some sort of oddball merge?

I think secure. had instructions about file ownership - looking...

https://secure.phabricator.com/book/phabricator/article/diffusion_hosting/

I think /var/repo should be owned by git:

The user the daemons run as. We'll call this daemon-user. This user is the only user which will interact with the repositories directly. Other accounts will sudo to this account in order to perform repository operations.

Yeah, logging perms should (I think) be fixed now. I was dumb when I chowned things and forgot what system users needed what access.

The release strategy of Phabricator was:

• everything goes into master asap, unless it's dangerous
• once a week, master gets merged into stable
  • after that, all the "dangerous" stuff lands to master
• important stuff that comes up during the week gets cherry-picked to stable.

This strategy was very simple, but made people upset that wanted numbered versions.

In T15000#289, @avivey wrote:

https://secure.phabricator.com/book/phabricator/article/diffusion_hosting/

I think /var/repo should be owned by git:

The user the daemons run as. We'll call this daemon-user. This user is the only user which will interact with the repositories directly. Other accounts will sudo to this account in order to perform repository operations.

I think that's set up properly, but for the log file location /var/log/phorge/ appears to be used by both the web server user and the vcs user. I also ran into similar issues using local filesystem storeage as that directory has to be accessible by multiple accounts.

Maybe create ssh.log file and chown it to git? and hope that's the only file it needs to write to?
I'm guessing from the name that it's only used by the SSH flow.

I don't remember ever running into this issue myself. I did have issues with the local filesystem storage and ownership.

I created Release Process for the release process.

Also to note, moving a wiki page to a new location seems to revert it to the default Space...

• move administrative SSH to port 2222

This one is going to require that everyone who currently has a cloned repo to update it, correct? I'll take a look later tonight at swapping this out, as the sooner the better IMO. I'll comment here before making the change.

Thanks @speck! I think we also need to update the NGINX config and phabricator.base-uri config to we.phorge.it from secure. Will also require updating the clone URI. You want to just bundle both changes at once to make things easier? Looks like @deadalnix already updated DNS so that should be hunky dory

I have some step by notes in our internal instance for getting SSH going - If you get stuck let me know and I will parse them into a public readable format

I see that it is working (for cloning anyway!)

@chris I'm looking to make the SSH configuration change shortly, having the administrative ssh go over port 2222 and vcs go over port 222. In the event everything goes horribly wrong does someone have physical access to this machine or some other control mechanism?

Hah yup, we're all good in case everything catches fire. I'm around all evening and can revert changes if anything goes haywire

The ports are switched

• Administrative port is now 2222
• VCS port is now 22

You will need to update your locally cloned repos to now remove the :2222 and update any ~/.ssh/config configurations you may have.

(I verified by starting a new ssh session over port 2222 and freshly cloning phorge after modifying diffusion.ssh-port)

I'm going to get aphlict up and running before looking at changing the domain name stuff. Not having notifications is kind of a bummer.

Whoops, commented on the wrong task, tested imagemagick in T15006#314

Notifications are also functional. Took me a minute to remember where the "test notification" feature is located (it's in your user settings > notifications)

Okay I'm going to try swapping out the URL for we.phorge.it. If everything goes well everyone will need to update their URLs and clone repos. If things don't go well I'll, uh, glue it back together

Okay I think everything is setup for the migration to we.phorge.it

• I added a port 80 configuration for we.phorge.it to nginx
• I ran certbot to grab a cert for we.phorge.it, I used --nginx
• I updated the nginx conf file to clean up the automatic modifications and setup secure.phorge.it and secure.phorge.dev to redirect to we.phorge.it
• I updated phabricator.base-uri to use we.phorge.it
• I updated notification.servers to use we.phorge.it
• I restarted nginx

Things seem to be working fine. I don't believe email needs configured at all after this. It even looks like running git fetch caused my local git repo's config file to update due to the redirect!

Infrastructure setup is being documented in server

• we.phorge.it works fine in Chrome, but arc has some issues w.r.t. CURLE_SSL_CACERT; I expect it might solve itself after a restart/update of my local machine.
• git fetch from the new uri shows no errors
• the push dragon still thinks rP51cb7a3db9 to 2abd75c162 is not fast-forward.

I'm going to have limited availability from now until Monday; I've updated D25000 to have just one commit on top of rP51cb7a3db9, so if someone wants to try pushing/landing, please do.

Something was funky in how the repo was originally imported that was causing the issues. Somehow got to a state where it wasn't properly a bare repo (there wasn't a working tree, but everything was still inside .git/ instead of the root folder). Not sure how that happened, but seems to be resolved now

It looks like Diviner was used to generate documentation however a lot of the documentation still refers to "Phabricator". We'll probably want a separate task just for reviewing and updating all the documentation to make sure it's appropriate.

Btw where is the source for the diviner books and how does it get generated?

(It's all in src/docs and can be generated with ./bin/diviner generate)

In T15000#408, @speck wrote:

It looks like Diviner was used to generate documentation however a lot of the documentation still refers to "Phabricator". We'll probably want a separate task just for reviewing and updating all the documentation to make sure it's appropriate.

Btw where is the source for the diviner books and how does it get generated?

T15012: Update Diviner documentation to reference Phorge and I’m about 70% done with the changes. I’m not just replacing “Phabricator” but I’m also removing references to paid support pacts. That’s why it isn’t a simple find and replace.

Oh, excellent! Thanks for looking into that.

For this phorge instance, I think we should configure auth providers to allow logging in with Github/Google etc.

And also we perhaps should have an IRC channel such as #phorge on Libera. It would be possible to bridge it to Zulip.

In T15000#888, @MacFan4000 wrote:

And also we perhaps should have an IRC channel such as #phorge on Libera. It would be possible to bridge it to Zulip.

There are seven people in there now, including me. As far as registering the channel goes, we won't get any help from staff unless we register as a project or a community. Having the channel would be very nice thing, as real-time chats are important IMO for projects like Phorge. (For support, etc) I would be more then happy to work with staff to get things set up.

I'm available to configure an inboud email receiver via a simple IMAP spooler, thanks to this:

T15086: Support Inbound Mail over IMAP

https://en.wikibooks.org/wiki/Phabricator_Administrator%27s_Handbook/Configure_inbound_e-mail/Using_an_existing_mailbox

I noticed some issues in D25129 probably related to our HTTP Auth (that very probably was never enabled). I added a checkbox about that.

https://we.phorge.it/book/phorge/article/diffusion_hosting/#configuring-http