Differential D25007 Diff 23 src/docs/contributor/reproduction_steps.diviner

Changeset View

Standalone View

src/docs/contributor/reproduction_steps.diviner

	@title Providing Reproduction Steps			@title Providing Reproduction Steps
	@group detail			@group detail

	Describes how to provide reproduction steps.			Describes how to provide reproduction steps.

	Overview			Overview
	========			========

	When you submit a bug report about Phabricator, you MUST include			When you submit a bug report about Phorge, you MUST include
	reproduction steps. We can not help you with bugs we can not reproduce, and			reproduction steps. We can not help you with bugs we can not reproduce, and
	will not accept reports which omit reproduction steps or have incomplete or			will not accept reports which omit reproduction steps or have incomplete or
	insufficient instructions.			insufficient instructions.

	This document explains what we're looking for in good reproduction steps.			This document explains what we're looking for in good reproduction steps.
	Briefly:			Briefly:

	- Reproduction steps must allow us to reproduce the problem locally on a			- Reproduction steps must allow us to reproduce the problem locally on a
	clean, up-to-date install of Phabricator.			clean, up-to-date install of Phorge.
	- Reproduction should be as simple as possible.			- Reproduction should be as simple as possible.

	Good reproduction steps can take time to write out clearly, simplify, and			Good reproduction steps can take time to write out clearly, simplify, and
	verify. As a reporter, we expect you to shoulder as much of this burden as you			verify. As a reporter, we expect you to shoulder as much of this burden as you
	can, to make make it easy for us to reproduce and resolve bugs.			can, to make make it easy for us to reproduce and resolve bugs.

	We do not have the resources to pursue reports with limited, inaccurate, or			We do not have the resources to pursue reports with limited, inaccurate, or
	incomplete reproduction steps, and will not accept them. These reports require			incomplete reproduction steps, and will not accept them. These reports require
	Show All 38 Lines
	observation is an obvious error like an exception, but can be important if a			observation is an obvious error like an exception, but can be important if a
	behavior is merely odd or ambiguous.			behavior is merely odd or ambiguous.


	Reliable Reproduction			Reliable Reproduction
	=====================			=====================

	When you file a bug report, the first thing we do to fix it is to try to			When you file a bug report, the first thing we do to fix it is to try to
	reproduce the problem locally on an up-to-date install of Phabricator. We will			reproduce the problem locally on an up-to-date install of Phorge. We will
	do this by following the steps you provide. If we can recreate the issue			do this by following the steps you provide. If we can recreate the issue
	locally, we can almost always resolve the problem (often very promptly).			locally, we can almost always resolve the problem (often very promptly).

	However, many reports do not have enough information, are missing important			However, many reports do not have enough information, are missing important
	steps, or rely on data (like commits, users, other projects, permission			steps, or rely on data (like commits, users, other projects, permission
	settings, feed stories, etc) that we don't have access to. We either can't			settings, feed stories, etc) that we don't have access to. We either can't
	follow these steps, or can't reproduce issues by following them.			follow these steps, or can't reproduce issues by following them.

	Reproduction steps must be complete and self-contained, and must allow			Reproduction steps must be complete and self-contained, and must allow
	anyone to reproduce the issue on a new, empty install of Phabricator. If			anyone to reproduce the issue on a new, empty install of Phorge. If
	the bug you're seeing depends on data or configuration which would not be			the bug you're seeing depends on data or configuration which would not be
	present on a new install, you need to include that information in your steps.			present on a new install, you need to include that information in your steps.

	For example, if you're seeing an issue which depends on a particular policy			For example, if you're seeing an issue which depends on a particular policy
	setting or configuration setting, you need to include instructions for creating			setting or configuration setting, you need to include instructions for creating
	the policy or adjusting the setting in your steps.			the policy or adjusting the setting in your steps.


	Getting Started			Getting Started
	===============			===============

	To generate reproduction steps, first find a sequence of actions which			To generate reproduction steps, first find a sequence of actions which
	reproduce the issue you're seeing reliably.			reproduce the issue you're seeing reliably.

	Next, write down everything you did as clearly as possible. Make sure each step			Next, write down everything you did as clearly as possible. Make sure each step
	is self-contained: anyone should be able to follow your steps, without access			is self-contained: anyone should be able to follow your steps, without access
	to private or proprietary data.			to private or proprietary data.

	Now, to verify that your steps provide a complete, self-contained way to			Now, to verify that your steps provide a complete, self-contained way to
	reproduce the issue, follow them yourself on a new, empty, up-to-date instance			reproduce the issue, follow them yourself on a new, empty, up-to-date instance
	of Phabricator.			of Phorge.

	If you can't easily start an empty instance locally, you can launch an empty
	instance on Phacility in about 60 seconds (see the "Resources" section for
	details).

				chrisUnsubmitted Done Inline Actions ref previous comment chris: ref previous comment
	If you can follow your steps and reproduce the issue on a clean instance,			If you can follow your steps and reproduce the issue on a clean instance,
	we'll probably be able to follow them and reproduce the issue ourselves.			we'll probably be able to follow them and reproduce the issue ourselves.

	If you can't follow your steps because they depend on information which is not			If you can't follow your steps because they depend on information which is not
	available on a clean instance (for example, a certain config setting), rewrite			available on a clean instance (for example, a certain config setting), rewrite
	them to include instructions to create that information (for example, adjusting			them to include instructions to create that information (for example, adjusting
	the config to the problematic value).			the config to the problematic value).

	If you follow your instructions but the issue does not reproduce, the issue			If you follow your instructions but the issue does not reproduce, the issue
	might already be fixed. Make sure your install is up to date.			might already be fixed. Make sure your install is up to date.

	If your install is up to date and the issue still doesn't reproduce on a clean			If your install is up to date and the issue still doesn't reproduce on a clean
	install, your reproduction steps are missing important information. You need to			install, your reproduction steps are missing important information. You need to
	figure out what key element differs between your install and the clean install.			figure out what key element differs between your install and the clean install.

	Once you have working reproduction steps, your steps may have details which			Once you have working reproduction steps, your steps may have details which
	aren't actually necessary to reproduce the issue. You may be able to simplify			aren't actually necessary to reproduce the issue. You may be able to simplify
	them by removing some steps or describing steps more narrowly. For help, see			them by removing some steps or describing steps more narrowly. For help, see
	"Simplifying Steps" below.			"Simplifying Steps" below.


	Resources
	=========

	We provide some resources which can make it easier to start building steps, or
	to simplify steps.

	Phacility Test Instances: You can launch a new, up-to-date instance of
	Phabricator on Phacility in about a minute at <https://admin.phacility.com>.
	These instances run `stable`.

	You can use these instances to make sure that issues haven't already been
	fixed, that they reproduce on a clean install, or that your steps are really
	complete, and that the root cause isn't custom code or local extensions. Using
	a test instance will avoid disrupting other users on your install.

	Test Repositories: There are several test repositories on
	`secure.phabricator.com` which you can push commits to if you need to build
	an example to demonstrate a problem.

	For example, if you're seeing an issue with a certain filename but the commit
	where the problem occurs is in a proprietary internal repository, push a commit
	that affects a file with a similar name to a test repository, then reproduce
	against the test data. This will allow you to generate steps which anyone can
	follow.


	Simplifying Steps			Simplifying Steps
	=================			=================

	If you aren't sure how to simplify reproduction steps, this section has some			If you aren't sure how to simplify reproduction steps, this section has some
	advice.			advice.

	In general, you'll simplify reproduction steps by first finding a scenario			In general, you'll simplify reproduction steps by first finding a scenario
	where the issue reproduces reliably (a "bad" case) and a second, similar			where the issue reproduces reliably (a "bad" case) and a second, similar
	▲ Show 20 Lines • Show All 69 Lines • ▼ Show 20 Lines
	============			============

	If you have an issue which you can't build reproduction steps for, or which			If you have an issue which you can't build reproduction steps for, or which
	only reproduces in your environment, or which you don't want to narrow down			only reproduces in your environment, or which you don't want to narrow down
	to a minimal reproduction case, we can't accept it as a bug report. These			to a minimal reproduction case, we can't accept it as a bug report. These
	issues are tremendously time consuming for us to pursue and rarely benefit			issues are tremendously time consuming for us to pursue and rarely benefit
	more than one install.			more than one install.

	If the issue is important but falls outside the scope of permissible bug
	reports, we're happy to provide more tailored support at consulting rates. See
	[[ https://secure.phabricator.com/w/consulting/ \| Consulting ]] for details.


	Next Steps			Next Steps
	==========			==========

	Continue by:			Continue by:

	- returning to @{article:Contributing Bug Reports}.			- returning to @{article:Contributing Bug Reports}.