Differential D25007 Diff 23 src/docs/user/configuration/storage_adjust.diviner

Changeset View

Standalone View

src/docs/user/configuration/storage_adjust.diviner

	@title Managing Storage Adjustments			@title Managing Storage Adjustments
	@group config			@group config

	Explains how to apply storage adjustments to the MySQL schemata.			Explains how to apply storage adjustments to the MySQL schemata.

	Overview			Overview
	========			========

	Phabricator uses a workflow called //storage adjustment// to make some minor			Phorge uses a workflow called //storage adjustment// to make some minor
	kinds of changes to the MySQL schema. This workflow compliments the //storage			kinds of changes to the MySQL schema. This workflow compliments the //storage
	upgrade// workflow, which makes major changes.			upgrade// workflow, which makes major changes.

	You can perform storage adjustment by running:			You can perform storage adjustment by running:

	phabricator/ $ ./bin/storage adjust			phorge/ $ ./bin/storage adjust

	This document describes what adjustments are, how they relate to storage			This document describes what adjustments are, how they relate to storage
	upgrades, how to perform them, and how to troubleshoot issues with storage			upgrades, how to perform them, and how to troubleshoot issues with storage
	adjustment.			adjustment.


	Understanding Adjustments			Understanding Adjustments
	===================			===================

	Storage adjustments make minor changes to the Phabricator MySQL schemata to			Storage adjustments make minor changes to the Phorge MySQL schemata to
	improve consistency, unicode handling, and performance. Changes covered by			improve consistency, unicode handling, and performance. Changes covered by
	adjustment include:			adjustment include:

	- Character set and collation settings for columns, tables, and databases.			- Character set and collation settings for columns, tables, and databases.
	- Setting and removing "Auto Increment" on columns.			- Setting and removing "Auto Increment" on columns.
	- Adding, removing, renaming and adjusting keys.			- Adding, removing, renaming and adjusting keys.

	Adjustment does not make major changes to the schemata, like creating or			Adjustment does not make major changes to the schemata, like creating or
	Show All 23 Lines
	issues.			issues.

	These interfaces report //Errors//, which are serious issues that can not be			These interfaces report //Errors//, which are serious issues that can not be
	resolved through adjustment, and //Warnings//, which are minor issues that the			resolved through adjustment, and //Warnings//, which are minor issues that the
	adjustment workflow can resolve.			adjustment workflow can resolve.

	You can also review adjustments from the CLI, by running:			You can also review adjustments from the CLI, by running:

	phabricator/ $ ./bin/storage adjust			phorge/ $ ./bin/storage adjust

	Before you're prompted to actually apply adjustments, you'll be given a list of			Before you're prompted to actually apply adjustments, you'll be given a list of
	available adjustments. You can then make a choice to apply them.			available adjustments. You can then make a choice to apply them.


	Performing Adjustments			Performing Adjustments
	======================			======================

	To perform adjustments, run the `adjust` workflow:			To perform adjustments, run the `adjust` workflow:

	phabricator/ $ ./bin/storage adjust			phorge/ $ ./bin/storage adjust

	For details about flags, use:			For details about flags, use:

	phabricator/ $ ./bin/storage help adjust			phorge/ $ ./bin/storage help adjust

	You do not normally need to run this workflow manually: it will be run			You do not normally need to run this workflow manually: it will be run
	automatically after you run the `upgrade` workflow.			automatically after you run the `upgrade` workflow.


	History and Rationale			History and Rationale
	=====================			=====================

	The primary motivation for the adjustment workflow is MySQL's handling of			The primary motivation for the adjustment workflow is MySQL's handling of
	unicode character sets. Before MySQL 5.5, MySQL supports a character set called			unicode character sets. Before MySQL 5.5, MySQL supports a character set called
	`utf8`. However, this character set can not store 4-byte unicode characters			`utf8`. However, this character set can not store 4-byte unicode characters
	(including emoji). Inserting 4-byte characters into a `utf8` column truncates			(including emoji). Inserting 4-byte characters into a `utf8` column truncates
	the data.			the data.

	With MySQL 5.5, a new `utf8mb4` character set was introduced. This character			With MySQL 5.5, a new `utf8mb4` character set was introduced. This character
	set can safely store 4-byte unicode characters.			set can safely store 4-byte unicode characters.

	The adjustment workflow allows us to alter the schema to primarily use			The adjustment workflow allows us to alter the schema to primarily use
	`binary` character sets on older MySQL, and primarily use `utf8mb4` character			`binary` character sets on older MySQL, and primarily use `utf8mb4` character
	sets on newer MySQL. The net effect is that Phabricator works consistently and			sets on newer MySQL. The net effect is that Phorge works consistently and
	can store 4-byte unicode characters regardless of the MySQL version. Under			can store 4-byte unicode characters regardless of the MySQL version. Under
	newer MySQL, we can also take advantage of the better collation rules the			newer MySQL, we can also take advantage of the better collation rules the
	`utf8mb4` character set offers.			`utf8mb4` character set offers.

	The adjustment workflow was introduced in November 2014. If your install			The adjustment workflow was introduced in November 2014. If your install
	predates its introduction, your first adjustment may take a long time (we must			predates its introduction, your first adjustment may take a long time (we must
	convert all of the data out of `utf8` and into the appropriate character set).			convert all of the data out of `utf8` and into the appropriate character set).
	If your install was set up after November 2014, adjustments should generally			If your install was set up after November 2014, adjustments should generally
	Show All 29 Lines

	As with most commands, you can add the `--trace` flag to get more details about			As with most commands, you can add the `--trace` flag to get more details about
	what `bin/storage adjust` is doing. This may help you diagnose or understand any			what `bin/storage adjust` is doing. This may help you diagnose or understand any
	issues you encounter, and this data is useful if you file reports in the			issues you encounter, and this data is useful if you file reports in the
	upstream.			upstream.

	In general, adjustments are not critical. If you run into issues applying			In general, adjustments are not critical. If you run into issues applying
	adjustments, it is safe to file a task in the upstream describing the problem			adjustments, it is safe to file a task in the upstream describing the problem
	you've encountered and continue using Phabricator normally until the issue can			you've encountered and continue using Phorge normally until the issue can
	be resolved.			be resolved.

	Surplus Schemata			Surplus Schemata
	================			================

	After performing adjustment, you may receive an error that a table or column is			After performing adjustment, you may receive an error that a table or column is
	"Surplus". The error looks something like this:			"Surplus". The error looks something like this:

	\| Target \| Error \|			\| Target \| Error \|
	\| --- \| --- \|			\| --- \| --- \|
	\| phabricator_example.example_table \| Surplus \|			\| phorge_example.example_table \| Surplus \|

	Generally, "Surplus" means that Phabricator does not expect the table or column			Generally, "Surplus" means that Phorge does not expect the table or column
	to exist. These surpluses usually exist because you (or someone else			to exist. These surpluses usually exist because you (or someone else
	with database access) added the table or column manually. Rarely, they can			with database access) added the table or column manually. Rarely, they can
	also exist for other reasons. They are usually safe to delete, but because			also exist for other reasons. They are usually safe to delete, but because
	deleting them destroys data and Phabricator can not be sure that the table or			deleting them destroys data and Phorge can not be sure that the table or
	column doesn't have anything important in it, it does not delete them			column doesn't have anything important in it, it does not delete them
	automatically.			automatically.

	If you recognize the schema causing the issue as something you added and you			If you recognize the schema causing the issue as something you added and you
	don't need it anymore, you can safely delete it. If you aren't sure whether			don't need it anymore, you can safely delete it. If you aren't sure whether
	you added it or not, you can move the data somewhere else and delete it later.			you added it or not, you can move the data somewhere else and delete it later.

	To move a table, first create a database for it like `my_backups`. Then, rename			To move a table, first create a database for it like `my_backups`. Then, rename
	the table to move it into that database (use the table name given in the error			the table to move it into that database (use the table name given in the error
	message):			message):

	```lang=sql			```lang=sql
	CREATE DATABASE my_backups;			CREATE DATABASE my_backups;
	RENAME TABLE phabricator_example.example_table			RENAME TABLE phorge_example.example_table
	TO my_backups.example_table;			TO my_backups.example_table;
	```			```

	Phabricator will ignore tables that aren't in databases it owns, so you can			Phorge will ignore tables that aren't in databases it owns, so you can
	safely move anything you aren't sure about outside of the Phabricator databases.			safely move anything you aren't sure about outside of the Phorge databases.

	If you're sure you don't need a table, use `DROP TABLE` to destroy it,			If you're sure you don't need a table, use `DROP TABLE` to destroy it,
	specifying the correct table name (the one given in the error message):			specifying the correct table name (the one given in the error message):

	```lang=sql			```lang=sql
	DROP TABLE phabricator_example.example_table;			DROP TABLE phorge_example.example_table;
	```			```

	This will destroy the table permanently.			This will destroy the table permanently.