Page MenuHomePhorge
Create Task
Maniphest T15371

RuntimeException in preg_replace_callback: Text disappears due to catastrophic backtracking regex in Remarkup parsing
Closed, ResolvedPublic
Actions

Description

Upstreaming from https://phabricator.wikimedia.org/T326398 , still valid.

  1. Phorge at 8d2b481bb5a83d1a4ce82014cabdc83fa3ca432d on PHP 8.2 (though also an issue in PHP 7)
  2. Go to an existing task to add a comment, or try to create the description in a new task
  3. In the Comment or Description field, write the line write {{#something:}} while watching the Comment/Description Preview
  4. Preview gets empty after finishing {{#something:}}
  5. If a new task, click Set Sail for Adventure: Nothing happens.

Error log shows:

[08-May-2023 22:03:49 UTC] [2023-05-08 22:03:49] EXCEPTION: (RuntimeException) preg_replace_callback(): Passing null to parameter #3 ($subject) of type array|string is deprecated at [<arcanist>/src/error/PhutilErrorHandler.php:261]
[08-May-2023 22:03:49 UTC] arcanist(head=renderDashboardTabPanelContent, ref.master=d47289622650, ref.renderDashboardTabPanelContent=d47289622650), phorge(head=master, ref.master=8d2b481bb5a8)
[08-May-2023 22:03:49 UTC]   #0 <#2> PhutilErrorHandler::handleError(integer, string, string, integer) called at [<arcanist>/src/error/PhutilErrorHandler.php:261]
[08-May-2023 22:03:49 UTC]   #1 <#2> preg_replace_callback(string, array, NULL) called at [<phorge>/src/infrastructure/markup/rule/PhabricatorObjectRemarkupRule.php:201]
[08-May-2023 22:03:49 UTC]   #2 <#2> PhabricatorObjectRemarkupRule::apply(NULL) called at [<phorge>/src/infrastructure/markup/blockrule/PhutilRemarkupBlockRule.php:92]
[08-May-2023 22:03:49 UTC]   #3 <#2> PhutilRemarkupBlockRule::applyRules(string) called at [<phorge>/src/infrastructure/markup/blockrule/PhutilRemarkupDefaultBlockRule.php:17]
[08-May-2023 22:03:49 UTC]   #4 <#2> PhutilRemarkupDefaultBlockRule::markupText(string, NULL) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:283]
[08-May-2023 22:03:49 UTC]   #5 <#2> PhutilRemarkupEngine::markupBlock(array) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:140]
[08-May-2023 22:03:49 UTC]   #6 <#2> PhutilRemarkupEngine::preprocessText(string) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:106]
[08-May-2023 22:03:49 UTC]   #7 <#2> PhutilRemarkupEngine::markupText(string) called at [<phorge>/src/infrastructure/markup/PhabricatorMarkupEngine.php:614]
[08-May-2023 22:03:49 UTC]   #8 <#2> PhabricatorMarkupEngine::extractPHIDsFromMentions(PhabricatorUser, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2059]
[08-May-2023 22:03:49 UTC]   #9 <#2> PhabricatorApplicationTransactionEditor::buildSubscribeTransaction(ManiphestTask, array, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2221]
[08-May-2023 22:03:49 UTC]   #10 <#2> PhabricatorApplicationTransactionEditor::expandSupportTransactions(ManiphestTask, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:1222]
[08-May-2023 22:03:49 UTC]   #11 <#2> PhabricatorApplicationTransactionEditor::applyTransactions(ManiphestTask, array) called at [<phorge>/src/applications/transactions/editengine/PhabricatorEditEngine.php:2031]
[08-May-2023 22:03:49 UTC]   #12 <#2> PhabricatorEditEngine::buildCommentResponse(ManiphestTask) called at [<phorge>/src/applications/transactions/editengine/PhabricatorEditEngine.php:998]
[08-May-2023 22:03:49 UTC]   #13 <#2> PhabricatorEditEngine::buildResponse() called at [<phorge>/src/applications/maniphest/controller/ManiphestTaskEditController.php:12]
[08-May-2023 22:03:49 UTC]   #14 <#2> ManiphestTaskEditController::handleRequest(AphrontRequest) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:284]
[08-May-2023 22:03:49 UTC]   #15 phlog(RuntimeException) called at [<phorge>/src/aphront/handler/PhabricatorAjaxRequestExceptionHandler.php:27]
[08-May-2023 22:03:49 UTC]   #16 PhabricatorAjaxRequestExceptionHandler::handleRequestThrowable(AphrontRequest, RuntimeException) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:751]
[08-May-2023 22:03:49 UTC]   #17 AphrontApplicationConfiguration::handleThrowable(RuntimeException) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:296]
[08-May-2023 22:03:49 UTC]   #18 AphrontApplicationConfiguration::processRequest(AphrontRequest, PhutilDeferredLog, AphrontPHPHTTPSink, MultimeterControl) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:203]
[08-May-2023 22:03:49 UTC]   #19 AphrontApplicationConfiguration::runHTTPRequest(AphrontPHPHTTPSink) called at [<phorge>/webroot/index.php:35]
[08-May-2023 22:03:52 UTC] [2023-05-08 22:03:52] EXCEPTION: (RuntimeException) preg_replace_callback(): Passing null to parameter #3 ($subject) of type array|string is deprecated at [<arcanist>/src/error/PhutilErrorHandler.php:261]
[08-May-2023 22:03:52 UTC] arcanist(head=renderDashboardTabPanelContent, ref.master=d47289622650, ref.renderDashboardTabPanelContent=d47289622650), phorge(head=master, ref.master=8d2b481bb5a8)
[08-May-2023 22:03:52 UTC]   #0 <#2> PhutilErrorHandler::handleError(integer, string, string, integer) called at [<arcanist>/src/error/PhutilErrorHandler.php:261]
[08-May-2023 22:03:52 UTC]   #1 <#2> preg_replace_callback(string, array, NULL) called at [<phorge>/src/infrastructure/markup/rule/PhabricatorObjectRemarkupRule.php:201]
[08-May-2023 22:03:52 UTC]   #2 <#2> PhabricatorObjectRemarkupRule::apply(NULL) called at [<phorge>/src/infrastructure/markup/blockrule/PhutilRemarkupBlockRule.php:92]
[08-May-2023 22:03:52 UTC]   #3 <#2> PhutilRemarkupBlockRule::applyRules(string) called at [<phorge>/src/infrastructure/markup/blockrule/PhutilRemarkupDefaultBlockRule.php:17]
[08-May-2023 22:03:52 UTC]   #4 <#2> PhutilRemarkupDefaultBlockRule::markupText(string, NULL) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:283]
[08-May-2023 22:03:52 UTC]   #5 <#2> PhutilRemarkupEngine::markupBlock(array) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:140]
[08-May-2023 22:03:52 UTC]   #6 <#2> PhutilRemarkupEngine::preprocessText(string) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:106]
[08-May-2023 22:03:52 UTC]   #7 <#2> PhutilRemarkupEngine::markupText(string) called at [<phorge>/src/infrastructure/markup/PhabricatorMarkupEngine.php:614]
[08-May-2023 22:03:52 UTC]   #8 <#2> PhabricatorMarkupEngine::extractPHIDsFromMentions(PhabricatorUser, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2059]
[08-May-2023 22:03:52 UTC]   #9 <#2> PhabricatorApplicationTransactionEditor::buildSubscribeTransaction(ManiphestTask, array, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2221]
[08-May-2023 22:03:52 UTC]   #10 <#2> PhabricatorApplicationTransactionEditor::expandSupportTransactions(ManiphestTask, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:1222]
[08-May-2023 22:03:52 UTC]   #11 <#2> PhabricatorApplicationTransactionEditor::applyTransactions(ManiphestTask, array) called at [<phorge>/src/applications/transactions/editengine/PhabricatorEditEngine.php:2031]
[08-May-2023 22:03:52 UTC]   #12 <#2> PhabricatorEditEngine::buildCommentResponse(ManiphestTask) called at [<phorge>/src/applications/transactions/editengine/PhabricatorEditEngine.php:998]
[08-May-2023 22:03:52 UTC]   #13 <#2> PhabricatorEditEngine::buildResponse() called at [<phorge>/src/applications/maniphest/controller/ManiphestTaskEditController.php:12]
[08-May-2023 22:03:52 UTC]   #14 <#2> ManiphestTaskEditController::handleRequest(AphrontRequest) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:284]
[08-May-2023 22:03:52 UTC]   #15 phlog(RuntimeException) called at [<phorge>/src/aphront/handler/PhabricatorDefaultRequestExceptionHandler.php:41]
[08-May-2023 22:03:52 UTC]   #16 PhabricatorDefaultRequestExceptionHandler::handleRequestThrowable(AphrontRequest, RuntimeException) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:751]
[08-May-2023 22:03:52 UTC]   #17 AphrontApplicationConfiguration::handleThrowable(RuntimeException) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:296]
[08-May-2023 22:03:52 UTC]   #18 AphrontApplicationConfiguration::processRequest(AphrontRequest, PhutilDeferredLog, AphrontPHPHTTPSink, MultimeterControl) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:203]
[08-May-2023 22:03:52 UTC]   #19 AphrontApplicationConfiguration::runHTTPRequest(AphrontPHPHTTPSink) called at [<phorge>/webroot/index.php:35]
[08-May-2023 22:03:52 UTC] [2023-05-08 22:03:52] EXCEPTION: (PhutilAggregateException) Encountered a processing exception, then another exception when trying to build a response for the first exception.
    - RuntimeException: file_exists(): Passing null to parameter #1 ($filename) of type string is deprecated
    - RuntimeException: preg_replace_callback(): Passing null to parameter #3 ($subject) of type array|string is deprecated at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:316]
[08-May-2023 22:03:52 UTC] arcanist(head=renderDashboardTabPanelContent, ref.master=d47289622650, ref.renderDashboardTabPanelContent=d47289622650), phorge(head=master, ref.master=8d2b481bb5a8)
[08-May-2023 22:03:52 UTC]   #0 <#4> preg_replace_callback(string, array, NULL) called at [<phorge>/src/infrastructure/markup/rule/PhabricatorObjectRemarkupRule.php:201]
[08-May-2023 22:03:52 UTC]   #1 <#4> PhabricatorObjectRemarkupRule::apply(NULL) called at [<phorge>/src/infrastructure/markup/blockrule/PhutilRemarkupBlockRule.php:92]
[08-May-2023 22:03:52 UTC]   #2 <#4> PhutilRemarkupBlockRule::applyRules(string) called at [<phorge>/src/infrastructure/markup/blockrule/PhutilRemarkupDefaultBlockRule.php:17]
[08-May-2023 22:03:52 UTC]   #3 <#4> PhutilRemarkupDefaultBlockRule::markupText(string, NULL) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:283]
[08-May-2023 22:03:52 UTC]   #4 <#4> PhutilRemarkupEngine::markupBlock(array) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:140]
[08-May-2023 22:03:52 UTC]   #5 <#4> PhutilRemarkupEngine::preprocessText(string) called at [<phorge>/src/infrastructure/markup/remarkup/PhutilRemarkupEngine.php:106]
[08-May-2023 22:03:52 UTC]   #6 <#4> PhutilRemarkupEngine::markupText(string) called at [<phorge>/src/infrastructure/markup/PhabricatorMarkupEngine.php:614]
[08-May-2023 22:03:52 UTC]   #7 <#4> PhabricatorMarkupEngine::extractPHIDsFromMentions(PhabricatorUser, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2059]
[08-May-2023 22:03:52 UTC]   #8 <#4> PhabricatorApplicationTransactionEditor::buildSubscribeTransaction(ManiphestTask, array, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:2221]
[08-May-2023 22:03:52 UTC]   #9 <#4> PhabricatorApplicationTransactionEditor::expandSupportTransactions(ManiphestTask, array) called at [<phorge>/src/applications/transactions/editor/PhabricatorApplicationTransactionEditor.php:1222]
[08-May-2023 22:03:52 UTC]   #10 <#4> PhabricatorApplicationTransactionEditor::applyTransactions(ManiphestTask, array) called at [<phorge>/src/applications/transactions/editengine/PhabricatorEditEngine.php:2031]
[08-May-2023 22:03:52 UTC]   #11 <#4> PhabricatorEditEngine::buildCommentResponse(ManiphestTask) called at [<phorge>/src/applications/transactions/editengine/PhabricatorEditEngine.php:998]
[08-May-2023 22:03:52 UTC]   #12 <#4> PhabricatorEditEngine::buildResponse() called at [<phorge>/src/applications/maniphest/controller/ManiphestTaskEditController.php:12]
[08-May-2023 22:03:52 UTC]   #13 <#4> ManiphestTaskEditController::handleRequest(AphrontRequest) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:284]
[08-May-2023 22:03:52 UTC]   #14 <#3> PhutilErrorHandler::handleError(integer, string, string, integer) called at [<arcanist>/src/error/PhutilErrorHandler.php:261]
[08-May-2023 22:03:52 UTC]   #15 <#3> file_exists(NULL) called at [<arcanist>/src/filesystem/Filesystem.php:1068]
[08-May-2023 22:03:52 UTC]   #16 <#3> Filesystem::pathExists(NULL) called at [<arcanist>/src/filesystem/Filesystem.php:1169]
[08-May-2023 22:03:52 UTC]   #17 <#3> Filesystem::assertExists(NULL) called at [<arcanist>/src/filesystem/Filesystem.php:1020]
[08-May-2023 22:03:52 UTC]   #18 <#3> Filesystem::isDescendant(NULL, string) called at [<phorge>/src/view/widget/AphrontStackTraceView.php:33]
[08-May-2023 22:03:52 UTC]   #19 <#3> AphrontStackTraceView::render() called at [<phorge>/src/view/AphrontView.php:222]
[08-May-2023 22:03:52 UTC]   #20 <#3> AphrontView::producePhutilSafeHTML() called at [<phorge>/src/infrastructure/markup/render.php:115]
[08-May-2023 22:03:52 UTC]   #21 <#3> phutil_escape_html(AphrontStackTraceView) called at [<phorge>/src/infrastructure/markup/render.php:139]
[08-May-2023 22:03:52 UTC]   #22 <#3> phutil_escape_html(array) called at [<phorge>/src/infrastructure/markup/render.php:97]
[08-May-2023 22:03:52 UTC]   #23 <#3> phutil_tag(string, array, array) called at [<phorge>/src/aphront/handler/PhabricatorDefaultRequestExceptionHandler.php:62]
[08-May-2023 22:03:52 UTC]   #24 <#3> PhabricatorDefaultRequestExceptionHandler::handleRequestThrowable(AphrontRequest, RuntimeException) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:751]
[08-May-2023 22:03:52 UTC]   #25 <#3> AphrontApplicationConfiguration::handleThrowable(RuntimeException) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:296]
[08-May-2023 22:03:52 UTC]   #26 <#2> AphrontApplicationConfiguration::processRequest(AphrontRequest, PhutilDeferredLog, AphrontPHPHTTPSink, MultimeterControl) called at [<phorge>/src/aphront/configuration/AphrontApplicationConfiguration.php:203]
[08-May-2023 22:03:52 UTC]   #27 <#2> AphrontApplicationConfiguration::runHTTPRequest(AphrontPHPHTTPSink) called at [<phorge>/webroot/index.php:35]
[08-May-2023 22:03:52 UTC]   #28 phlog(PhutilAggregateException) called at [<phorge>/src/aphront/response/AphrontUnhandledExceptionResponse.php:32]
[08-May-2023 22:03:52 UTC]   #29 AphrontUnhandledExceptionResponse::setException(PhutilAggregateException) called at [<phorge>/webroot/index.php:46]

Revisions and Commits

rP Phorge
D25838rP9c73d62c4466 Rewrite regex for project names to be not prone to catastrophic backtracking

Related Objects

Event Timeline

aklapper created this task.May 9 2023, 07:52
Herald added subscribers: Cigaryno, Matthew, chris and 2 others. · View Herald TranscriptMay 9 2023, 07:52
valerio.bozzolan added a project: Affects-Wikimedia.May 9 2023, 08:02
valerio.bozzolan added a project: Remarkup.
valerio.bozzolan subscribed.
valerio.bozzolan added a comment.May 9 2023, 08:05

Premising that very probably a fix there would not 100% resolve that upstream issue but surely this deserves a fix :D

avivey edited projects, added PHP 8 support; removed Remarkup, Phorge.May 9 2023, 10:35
aklapper removed a project: PHP 8 support.May 9 2023, 13:37

This is unrelated to PHP8. This might turn out to be a duplicate/variation of T15372.

valerio.bozzolan added a project: Remarkup.EditedMay 9 2023, 14:09

(Sorry for the ping poing, re-adding Remarkup hoping to be useful since this seems an issue in Remarkup itself, but last word to avivey)

avivey added a project: Bug Reports.May 9 2023, 15:19
aklapper added a comment.Aug 5 2024, 12:44

This is an issue in https://we.phorge.it/source/phorge/browse/master/src/infrastructure/markup/rule/PhabricatorObjectRemarkupRule.php . In getObjectEmbedPattern(), preg_quote($prefix) returns \# and the method finally returns (\B{\#([^.\s?!,:;{}#\(\)"'\*/~]+(?:[^\s?!,:;{}#\(\)"'\*/~]*[^.\s?!,:;{}#\(\)"'\*/~]+)*)([,\s](?:[^}\\]|\\.)*)?}\B)u.
Within apply($text), $text becomes null after that first preg_replace_callback, so the second call barks.

aklapper added a comment.Oct 25 2024, 10:19

This exception happens once $rule in the loop foreach ($this->getMarkupRules() as $rule) in PhutilRemarkupBlockRule::applyRules($text) becomes ProjectRemarkupRule. That's where it blows up.

This only seems to happen if the "readable" string (translation in this case) has 9 or more characters.

In ProjectRemarkupRule, adding : to $never_edge (and running ./bin/cache purge --caches remarkup) doesn't make a difference either.

pppery subscribed.Sat, Nov 23, 20:18

I monkeypatched the code to print preg_last_error when the problematic preg_match returns null. The error is "Backtrack limit exhausted".

Sigh PHP ...

pppery added a comment.Sat, Nov 23, 20:20

Indeed https://devina.io/redos-checker says the aformentioned regex is exponential time worst-case

pppery added a comment.Sat, Nov 23, 20:30

Disentangling that monster regex, step 1:

(...)u is the delimiter and flags syntax, so (...)u means "Run the regex ... in Unicode mode" and the real regex is \B{\#([^.\s?!,:;{}#\(\)"'\*\/~]+(?:[^\s?!,:;{}#\(\)"'\*\/~]*[^.\s?!,:;{}#\(\)"'\*\/~]+)*)([,\s](?:[^}\\]|\\.)*)?}\B

I can reproduce the catastrophic backtracking behavior if I paste that into regex101.com and test it on the string {{#translation:}}.

pppery added a comment.EditedSat, Nov 23, 20:33

Step 2: Remove irrelevant character class exclusions:

\B{\#([^:]+(?:[^:;]*[^:]+)*)([,\s](?:[^}\\]|\\.)*)?}\B

The problem is (?:[^:;]*[^:]+)*. That means "any number of occurances of any number of non-colon characters followed by at least one non-colon characters". translation can fix into that loop in many different ways. No matter what way it chooses, it will fail to find the } at the end, so have to backtrack and try another.

pppery added a comment.Sat, Nov 23, 20:45

Digging deeper, the problem is specific to ProjectRemarkupRule::getObjectIDPattern. That returns:

[^<never_edge><never>]](?:[^<never>]*[^<never_edge><never>]+])*

Even if never is ab and never_edge is c then that produces [^abc](?:[^ab]*[^abc]+)*<more stuff>, which is exponential in the size of the project name as long as more stuff fails to match per https://devina.io/redos-checker

pppery added a revision: D25838: Rewrite regex for project names to be not prone to catastrophic backtracking.Sat, Nov 23, 21:08
pppery renamed this task from RuntimeException in preg_replace_callback: Text disappears due to Remarkup parsing to RuntimeException in preg_replace_callback: Text disappears due to catastrophic backtracking regex in Remarkup parsing.Sat, Nov 23, 21:17
pppery claimed this task.
pppery closed this task as Resolved by committing rP9c73d62c4466: Rewrite regex for project names to be not prone to catastrophic backtracking.Thu, Dec 5, 17:40
pppery added a commit: rP9c73d62c4466: Rewrite regex for project names to be not prone to catastrophic backtracking.
valerio.bozzolan mentioned this in Next Up.Tue, Dec 10, 14:03
Content licensed under Creative Commons Attribution-ShareAlike 4.0 (CC-BY-SA) unless otherwise noted; code licensed under Apache 2.0 or other open source licenses. · CC BY-SA 4.0 · Apache 2.0