Page MenuHomePhorge

Figure out if there are patches from Wikimedia's fork that are desirable to upstream in Phorge
Open, LowPublic

Description

Over the years (since 2015) of maintaining Wikimedia's fork of Phabricator, I made numerous small changes which weren't upstreamed. Anything that seemed obviously general was upstreamed but most things were not. Many of these patches are specific to Wikimedia's use of Phabricator, however, there are likely to be some useful things that should find their way into Phorge.

One potential situation is patches where upstream Phabricator developers specifically objected to the change but where perhaps the Phorge community has a different opinion, more in line with Wikimedia's perspective than with Phabricator. I don't know if this will be the case but some of the changes might be worth at least considering.

For starters I intend to go through the list (P3) and identify anything that I feel would be potentially useful, then for each promising patch I'll either propose a new differential revision against phorge or I'll ask via other communication channels to seek feedback from the community.
Full list of Wikimedia patches:

Legend: Wikimedia commits highlighted in yellow already have a sub-task:

167d87426e1de (HEAD -> wmf/stable, tag: release/2022-02-03/1, origin/wmf/stable) Don't show default status icon on workboard cards (fixes T297249)
24f4cae8e183d Limit task merges to 5 tasks per transaction (fixes T298063)
3ebfe30890b52 Add column sequence to the conduit api results for column.search
400b4ad5fac93 add task phid to the response in ManiphestProjectTaskTransactionsConduitAPIMethod
5dd7aec132de3 Fix links to task search by age from the project reports page
6a7121afcbfaa Add status icons to workboard tiles.
744a94dc04b3f Fix validation of "column" transaction type in "maniphest.edit"
8c0b503779bde Add status and isDefaultColumn to the project.column.search api
910b91709c37f (tag: release/2021-07-14/1, tag: release/2021-06-23/1, ProjectTaskTransactionsConduitApiMethod) add ManiphestProjectTaskTransactionsConduitAPIMethod for #DataDataData
108a6a445c1608 (origin/T284397, T284397) Debug T284397
11f2fd14dc1ede Add metadata to the gettasktransactions conduit method response
125e1ebc4d23e4 change mode to match production
13f75c9e53e9bf Revert "WMF Hack: Prompt to replace auth factors from before August 2019"
147246e8ae7d1e Updated login sprite sheet
15c9c815f08005 Fix hidden top-border on task type badges
160f94052396d1 New MediaWiki logo for the icon that represents the MediaWiki oauth provider.
177e4f4b7a03d3 Fix miscount of age histogram and add links to date range search
1856f00f7e0213 Project Metrics - add links to search for specific columns.
192d0f05fe0331 Update celerity map
209082e93c588d Fix undefined index error in project.search conduit queries
214fac3fde9539 (tag: release/2021-03-24/1) Improve labels for report bar chart.
227d0aa6596286 Add more metrics to project reports
23de1ec2c9e1dc Adds a countdown.create capability to limit countdown creation.
244faa17d0a295 Fix isValidGitShallowCloneResponse
257a26c53925e0 Revert "Fix isValidGitShallowCloneResponse"
26620fa0816bfc Fix isValidGitShallowCloneResponse
27ff7025ef62e2 (ProjectMetricsCountOlderTasks) Show 'older' tasks on project metrics task age chart.
28f3db55b19584 (gitlab/MetricsMergeRequest, MetricsMergeRequest) Mostly clean up code formatting
2926c7463ede27 Phabricator Project Reports
302c93d69e42d2 Convert instances of PhutilSafeHTM to plain strings
316826fb6bc09c (tag: release/2020-09-10/1) liberate
324e9fe6e8a314 Conduit method for legalpad signatures
33c7fd8fef1a48 Only add rawCorpus if there are ferret tokens
349a28f5e683dd Ran arc liberate
359d553b77d0f8 (tag: release/2020-06-25/1) Set cluster.search config option to unlocked
3616188b81fa84 add wikimedia extensions css file and compile with celerity
378282af06e894 (tag: release/2020-05-14/1) Don't display user 'blurb' for disabled accounts.
38909fe0be8ef8 Use hasApplicationCapability instead of PhabricatorPolicyFilter::hasCapability
391ff79a37e3a3 Require ProjectCreateProjectsCapability::CAPABILITY in order to archive projects
40042685bdf637 Allow milestone columns to be reordered.
4163c5e293cf0d WMF: expert-mode: show id field in maniphest search form
42b3a61693ced4 Fix half-baked expert-mode code
43e95157e39bf5 Show matching context from the document body in ferret search results
4482a0d4251ae6 (tag: release/2020-02-13/1) Don't escape the html of atom link tags
451924c95c8e71 Ran `arc liberate`
462148f5495c8d fixup the display of the subtype tag on task lists
478a99d78f5b88 Move the construction of PhabricatorFulltextResultSet into the Fulltext Storage Engine
489a65bc28eca2 (tag: release/2020-01-23/1) Escape quotes in elasticsearch queries
492785d6a8b210 WMF Hack: Prompt to replace auth factors from before August 2019
5060759f2ef9d5 PhabricatorSearchService::newResultSet should return PhabricatorFulltextResultSet
513ed56bdd557d expert-mode: Always show phid fields in query forms
523d33d1cceac7 Implement Atom/RSS discovery on Phame blog pages
53f166f91d760f (tag: release/2019-12-19/1) update phutil library map (`arc liberate`)
54c78732e954a4 rawurlencode the email address
55efc7de14fe4a Add cc to mailto: links in maniphest.
5683f6b43ddff3 WMF Local hack: Add $viewer arg to newTagView()
57aa43d3d745a3 Fix fatal exception in PhabricatorProjectListView
58aeceaf96333e Work around rendering engine freeze in Chrome 77 affecting workboards
59db569a45a8da Update library map - missing class PhabricatorWorkboardViewState
60ce7e7da88f5b (tag: release/2019-08-14/1) Don't load subtype map if there are no subtype transactions.
61f8b44b65514e (tag: release/2019-05-08/1) Added move_project script from upstream.
62d7744d7e9bd0 Fix fatal in phab calendar
63a54b3cd94e94 (tag: release/2019-03-20/1) git import: only match author identities when email is exact match
642b3156af9a94 Show active subprojects before archived subprojects
65e05b36986426 (tag: release/2019-02-27/1) Comply with upstream type hint by returning an array
669a10d3e771be (stable) Fix URI construction of typeahead browse "more" pager
67b1a26d919cb1 (tag: release/2019-02-20/1) Revert "Stop using _type (Elastic 6.0 support)"
6883e4d171aa07 Update celerity map, phutil library
693feb2d96899e Phabricator, now with moar stroopwafels.
70bba62cf52435 Hide the "hidden" fields on custom form previews
71aec846f37b26 Fix the check for is_admin in LegalpadDocumentRequireSignatureTransaction
720c864b875f67 Stop using _type (Elastic 6.0 support)
736553a9dbb8a5 Fix merge error in 20180208.maniphest.02.populate.php
748cbf206d35de (tag: release/2018-08-23/1, tag: release/2018-08-08/1, merge-upstream) Make 'open tasks' appear by default on project profile menus
759191d4838278 Make "task type" and date-type custom fields work in herald.
76386b86fde376 Add content-security-policy header rule for commons.wikimedia.org
773443ae549058 Fix spelling in config option refs T199848
785577304cd98d Remove 'include_in_all' for es6 compatibility.
79640c5beae64f Don't rate limit when the limit is set to zero.
800716d5f927b3 (tag: release/2018-07-11/1) Whitelist connections rather than just zero-rating them
8149ad8413aec8 Whitelist WMF and WMDE office IPs from rate limiting
82034df81669a7 Add a hook for oauth providers to veto logins
8311134c1dbca0 Exempt bot users from rate limiting.
8462391947d489 unlock favicon config option
855853b7357aa7 Add a --limit argument to the `workers` management cli
86bed5db237689 Ignore refs/changes/**/**/meta
87447032e1dac6 Add alter table statements to the migration
88ae334e4589a0 (tag: release/2018-04-12/1) Add custom task deadline display on workboard cards.
893de500bfc845 (tag: release/2018-03-07/1) Include post publish date in phame atom feeds
90f177f92217c2 (tag: release/2018-02-15/1) Remove 20180208.maniphest.01.close.sql, schema change already applied
914ca0e44bf025 disable the 20180208.maniphest.02.populate.php migration
92ecb8dbb02a88 (tag: release/2018-01-17/1) Lower the threshold for translations to be selectable in settings.
93e90d8d56dd3e Push Log Garbage collector: Flatten array before passing to queryfx
9412b9d17b042f Only hide the login form if ldap is enabled.
959d64cc2cb00a Prevent hiding the PhabricatorProjectDetailsProfileMenuItem
96e336f8547c9f Push logs: Only show IPs to admins and account owners.
973f8570e7a114 Adds a Garbage collector for Push Events and associated Push Logs.
9810c7e1139996 -isAdmin() +getIsAdmin()
99697a557d61ea Obfuscate the last 3 octets in push log.
10040d2742518cb Remove the distracting tableflip figlet from log messages.
101a9613b44b70c Remove function_score from query
10262d1fa986be4 Allow callsign to be null in PhabricatorRepositoryQuery
103c7e72b6c2462 (optimization) Remove extensions/ directory
10496701c4546bd Hide the LDAP login form by default, click to show
105e374b1a5a50a Boost recent documents in search results
106d606822dcce0 Fix the shouldHideByDefault hack
107893664bd44b8 EditEngine: 'Duplicate Form' action to create new forms from existing config.
1081a265de83e15 Change default field visibility to hidden
1096b116698ce9b Add the repository name to revisions in Differential list view.
11063ae55ee918a Run DiffusionRepositoryExtension hooks before canonical redirects
111853a5e1d409a Add support for DiffusionRepositoryExtension so we can support go-get in diffusion.
11252329ae09031 Slightly more printer-friendly css.
11389d7b6631f26 Add goat token
11425a70c827bdf Fix Undefined class constant
115970258dcf56d Support allow PHURL tags to use dash
1166baa99f72fa7 (arcpatch-D716) Lower number of MetaMTA failures from 250 to 100
117e7135d411890 Add back support for CustomGithubDownloadLinks
1187045f9808ddc call setobject before shouldEnableForRole
11951e0045c4506 Catch exception from invalid URIs passed to PhutilURI
120cfd6daf92077 Check for empty array in PhabricatorCalendarNotificationEngine::sendNotifications
121678062f1826f Add support for embedding videos from commons into remarkup
1228ad46e71239e initialize $phids to an empty array
1236a477423d7c6 Key the array on PHID
124656366f289d3 Make search highlights work with upstream changes
1251c30be2d664a Make sure that content returned by elasticsearch is properly escaped
126e5b9e045f3d5 More subtle styling of the highlighted search result matches
1278b86c654a125 highlight search.
1283af2dd4d1994 (T8285) Don't apply offset to elasticsearch query
1290b997aa85c0d fresh celery
130a8c393225646 Add highlighting support to Elasticsearch fulltext engine
131673bcdbd5ac4 Remove unused PhabricatorFulltextStorageEngineAggregate
1324ce052861c52 Make sure writes go to the right cluster
133d63dd2b53d78 Don't use $future->write but do set the content-type
134d9ad6be053cf Set content-type to application/json
1354313f1c06e19 Support multiple fulltext search clusters with 'cluster.search' config
13655e49eb2d5ad Make setService compatible with superclass method.
13758ae2dfbf6b5 Support multiple fulltext search clusters with 'cluster.search' config
138178a5478e4f5 Sync with upstream 'fulltext search cluster' patch
139f57d4cf81cc3 Support multiple fulltext search clusters with 'cluster.search' config
14029501cfafaae Support multiple fulltext search clusters with 'cluster.search' config
141da151e516dd0 Add missing class PhabricatorProfileMenuItem to __phutil_library_map__.php
142d2f8fe6e4e79 Fix setup warning on elasticsearch 5.0
14302002ff59b58 Make search.elastic.version an int not a string
14456e10336461f Add a domain whitelist to Phurl
145c0a409290f44 Use #events instead of #general-events
146c3b85644a29d Filter default calendar views by #general-events project.
1470f04f9dc3b32 more fulltext changes
148cd85f6cc7ba6 WIP: Refactoring the fulltext storage engine interface.
149c517fa37615b WIP: Refactoring the fulltext storage engine interface.
150ae2da20fa6c4 (arcpatch-D553) Add missing setViewer call in macro
151b6fb2c21dd06 More improvements to the elasticsearch storage engine
152659b5d7ccad5 Clean up "reorder" permissions in MenuEngine for personal favorites
1539ed2913e75b4 Prevent requesting too many records from elasticsearch.
154b09975a8d70d Fix wrong variable name in PhabricatorProjectProfileMenuEngine
15556caf6f92f7e Strip array keys before sending field list to elasticsearch
156d3017c715db2 Fix undefined PANELKEY const
1572b7ac61da43a add back some debug logging :)
15827ff9966d309 check user setting before global config.
1594252f2af7ab3 remove more debug logging.
1604ba9d95bc22f remove phlog() debugging
161ba74ee9373a6 (origin/elastic) Add loadEngineForViewer to PhabricatorFulltextStorageEngine
16236e31cf986b8 Fixed up the query builder.
16337e755fe9180 working towards a competent elasticsearch backend
1643eb1f5841bef Remove Edit Maniphest Tasks from diffusion
165f85036a1f2f6 Fix fatal with custom github links
166d8dd5f4b71ea Don't fail when all almanac cluster services are closed
1671c6e4523c8e1 patch to unstem queries
16829691f5c9b87 Elasticsearch query optimization + profiling
169bac71ddaca47 Separate config key 'search.elastic.enabled'
170a947b022d7d7 Drasticly improve performance when importing large repos
17112fb4697478a WmfConfigSource moved to extensions rPHEX
172e4912a175b12 putenv not setenv
173092f06197951 Revert "WmfConfigSource: Attempt to load a readable config file in all cases"
1742ecf79b4ea31 use wmf colors on notification favicons
1758dea3b34df79 set PHABRICATOR_ENV=phd in commit hook
176b88502e7c3f2 WmfConfigSource: Attempt to load a readable config file in all cases
177c55f5d5c059a (stable) Make project token sorting and normalization a little less hacky
178ea163216b0dc (stable) Fix an EditEngine issue with unlocking fields which can't be locked
179ece5ade0643c Revert "Hotfix "Undefined variable: field" in PhabricatorEditEngineConfiguration"
180225400ae55af make file mode match the mode puppet enforces
1819dff87c5e252 Hotfix "Undefined variable: field" in PhabricatorEditEngineConfiguration
182b5bcd9123b50 doh! fix syntax error
18334fe4154d403 Include the PHABRICATOR_ENV in ssh auth scripts
1849e9b2d958736 Hotfix: limit project_datasourcetoken joins to 5 tokens
185b88caa1a9e1b Revert "Use persistent connections to mysql"
186f38f9e13fd69 Infrastructure to support https://gerrit.wikimedia.org/r/#/c/321654/
18741a7167f9cfb WmfConfigSource - load environment-specific json configs
188fcbcc249d694 Use persistent connections to mysql
1898f7341ee603d remove PortStemmer.
190902ade9eba61 Filter search input that innodb doesn't like
1912393fdb13e12 Change permission on scripts to match production
192ab7ec99e899e Fix InnoDB Fulltext Search: Prepend + to each word in the query
193dbc2ad3debdf Fix error in AphrontFormDateControl
194f477ca9bc79c Fix searching for one letter or one number in elasticsearch
1951a9be39e825e Fix support for elasticsearch 2
196242a102bcd9b Fix missing $version variable declaration
1975347029702ef Supports any value higher then 2 for indexExists to work
198af4367086d9a Add support for elasticsearch 2
199702bd656d640 Add accesskey="F" to the search menu
2001667c5b2bf9f (arcpatch-D367) Expose the field text to maniphest advanced search again
201bf75469a3427 Check for empty output from git ls-tree
2023d1f1a8324b3 Fix Argument 4 passed to ManiphestTaskDetailController::buildPropertyView() must be of the type array
203023824c30a6b Revert "beautify repository search result view"
2042f26eb10330f Revert "Fix inconsistent behavior in calendar recurrence sequence, add sequence nav buttons"
20541c8bb0cec69 add needed config entry metamta.can-suppress-mail
2068b7abb141a18 Support suppressing email from select users.
207b842155d5e00 Increase the number of tags shown for each task in ManiphestTaskListView.
208acb736547c65 Don't show commits twice in task details.
209cf12fdf248df Fall back to parent tasks / subtasks when the task graph is big
21021f5f3dae953 For now, hard limit task graph at 100 nodes
211a1304fc3756c Include more projects on user profiles
2129f143f8321b9 (arcpatch-D16243) Ensure files live as long as intended.
213de19094b2d61 Updated built-in project profile icons.
2146a8722f2da78 return uri object
215be4e3a33a57f Prefer returning a http uri from getCloneURIObject
21654c0674cd613 (stable) Fix a flipped higlight vs no-highlight condition
217a49ab149303b (stable) Fix an XSS issue where Diffusion files exceeding the highlighting byte limit were not properly escaped
218e172510f5759 Wrap really long text properly in diffs
219ff38787beb5c Fix inconsistent behavior in calendar recurrence sequence, add sequence nav buttons
2204806bb240cdb add callsign to arcconfig
2213effc90afa31 Blacklist refs/cache-automerge, refs/notes and refs/drafts
222bf7eb8a42d64 (arcpatch-D260) Add support for parsing refs/meta/ and refs/drafts/
223d29badbfc756 beautify repository search result view
224f4099e2a9465 fix lint errors
2254445f9afbd2f Fix refs/changes/ not being parsed
2268167171f191b Fix typo
227152da2d872c8 This adds support for downloading from our github mirror.
228487c8db620fc Load PhabricatorMilestoneNavProfilePanel on milestone projects
22972b2dc1ef0b6 Add more repository details to harbormaster build steps
2306982a924cd5c fix capitalized Header function call
231190e2a1381c4 adjust default arcanist revision base rules
232dd7856c0e116 (arcpatch-D231) Improve differential tabs
233ecb85c8350fc Use monospace font in diffusion readme views
2349b4555cc69fe Prevent sending a header with unsafe characters [\r\n\0]
23544d190713781 allow tracking refs/changes
23646edd075aeae Don't autoclose refs/changes/
2372641d271ea41 Add support for importing refs other than branches/tags
2386b9b0cc5b26a Fix diffusion raw file view for php files.
239d04c16c61b61 remove phlog
240586aaa547ade add html+data for feed.query conduit api
2418ebb261f4cbe Cleaner hack for DiffusionCommitController
2420f5d5e87b5c7 [wmf] Add a barnstar and a cookie token to tokens spritesheet
2437e295c76ba26 hotfix D15306
24455ed8751cebc wmf hack hide acl* projects on user profiles.
2453e7c0d806ccf Improve subproject/milestone error handling for users who can't create projects
2469d372ca91f49 ran arc liberate
247cde0b9958ede Revert "[PATCH] Allow assigning users other than oneself when closing owner-less tasks"
248c0dda189f69c [PATCH] Allow assigning users other than oneself when closing owner-less tasks
249e0da181aface Add a bunch of apps and expensive URLs to robots.txt exclusion
250482c9869b428 add /files/ to robots.txt exclusion
251ee4be9c73d54 Add /multimeter/ to robots.txt exclusion
2522a4583d38715 Support custom fields in diffusion commit property views.
2532d2788ef94c2 WMF local hotfix for float:left bug in phabricator redesign-2015
254da55dd0355ac update celerity map
255d0f6b1450c21 wikimedia logo
2562dad469eca22 Add hard stops on empty batch edit sets
257c944d47e6a0b Add wikimedia logo for phabricator
25867178ef83c3d Allow different policy rules for different types of objects
259cace777ba0db Fail gracefully on bad API call when building commit mail
26085c4c54a2122 Hide email addresses on tasks submitted via email
261032e5f82ac81 Custom calendar icons.
262b07d0d4db87f Custom favicon for phabricator.wikimedia.org

Feel free to highlight other rows / create more sub-tasks.


We encourage the downstream discussion (https://phabricator.wikimedia.org/T333885) to participate here.

Related Objects

Event Timeline

Features unique to wikimedia's fork

  • e95157e39bf5 Show matching context from the document body in ferret search results
    • This is not perfect but it generally works - display a snippet from the matched document and highlight the matched words - kind standard and expected from any full-text search engine. Probably not my best work as far as code quality / it's a bit hacky and performance may not be stellar, however, it's been in use at wikimedia for quite some time without any major issues that I'm aware of.
  • bba62cf52435 Hide the "hidden" fields on custom form previews.
    • This is a single line change to css that makes things a lot cleaner when you have a lot of custom forms with a lot of custom fields.
    • submitted as D25037
  • 9191d4838278 Make "task type" and date-type custom fields work in herald.
  • 3d33d1cceac7 Implement Atom/RSS discovery on Phame blog pages
  • ebfe30890b52 Add column sequence to the conduit api results for column.search
    • This seems like an obvious omission from the conduit api for columns and the change is straightforward.
    • submitted as D25038
  • 893664bd44b8 EditEngine: 'Duplicate Form' action to create new forms from existing config.
    • This makes it much easier to clutter up your custom forms with 100 variations of your forms. It also makes it much easier to make a new form vs. starting from scratch every time. It's sort of a hack and the custom form management UI needs a lot of improvement generally, this was just the minimum change I could implement to make life slightly easier for myself and fellow Wikimedia phab admins. Not sure it's a good idea in the upstream without further changes to go with it.
  • 44a94dc04b3f Fix validation of "column" transaction type in "maniphest.edit"

Minor Security Hardening:

  • 4f4cae8e183d Limit task merges to 5 tasks per transaction (fixes T298063)
  • 1ff79a37e3a3 Require ProjectCreateProjectsCapability::CAPABILITY in order to archive projects
    • I believe this is a sane security improvement

Behavior / Product changes that were not wanted in Phabricator upstream:

  • ecb85c8350fc Use monospace font in diffusion readme views
    • I think I just forgot to propose this upsteream
  • 042685bdf637 Allow milestone columns to be reordered.
    • epriestley was very much against this idea but wikimedia's users loved it.
    • Milestones, unlike subprojects, are mutually exclusive, so a task can only be in one milestone in the superproject. This has the convenience that adding a task to a milestone automatically removes it from any sibling milestone within the same superproject.
    • Reordering milestones is convenient when you want to treat milestones as workflow steps rather than sequential numerical versions. Example: https://phabricator.wikimedia.org/project/board/20/
    • upstream product vision was to use milestones exclusively for sequential things like "Maniphest V2" or sprints.
    • Wikimedia chose to abuse milestones for workflow steps and being stuck with the ordering that phabricator enforces was a big inconvenience for that use-case.

Maybe the milestone reordering feature could be behind an opt-in toggle? Personally I don't see any problem with that and I'm also not sure why milestones have to be strictly sequential in the first place. I found myself creating milestones accidentally out of sequence and the only way to "reorder" was renaming the milestones, which felt quite punishing.

I found myself creating milestones accidentally out of sequence and the only way to "reorder" was renaming the milestones, which felt quite punishing.

Me too. So we may love this as compromise: T15143: Allow to quick-edit a Milestone name from Workboard menu

The next candidate for discussion could be probably this one:

valerio.bozzolan updated the task description. (Show Details)

Here's another one that I think deserves to be upstreamed: https://phabricator.wikimedia.org/T230787 adds context and search term highlights to fulltext search results. Currently Phorge and upstream Phabricator only show matching document titles with highlights on keywords in the title but not the body.
The efficiency of my solution is questionable, however, it's working well enough for Wikimedia's use of the feature.

T15694: Consider adopting the "search results context highlighting" feature from Wikimedia's fork

Another good simple candidate GDPR-friendly:

Obfuscate the last 3 octets in push log if you are not Administrator

https://phabricator.wikimedia.org/rPHAB697a557d61ea32aa53e574752b8a1e726c16483c