Page MenuHomePhabricator

Botched Revision Data
Open, HighPublic

Description

It's been true for several days, but ...

Attempting to edit any existing page results in a traceback. For example:

[98fedcb8f435b46ad5d64ace] /w/index.php?title=List_of_wikis&action=edit
Error from line 1151 of /var/www/live/w/includes/EditPage.php: Call to a member function getModel() on null

Backtrace:

#0 /var/www/live/w/includes/EditPage.php(691): EditPage->initialiseForm()
#1 /var/www/live/w/includes/actions/EditAction.php(55): EditPage->edit()
#2 /var/www/live/w/includes/MediaWiki.php(511): EditAction->show()
#3 /var/www/live/w/includes/MediaWiki.php(302): MediaWiki->performAction(Article, Title)
#4 /var/www/live/w/includes/MediaWiki.php(854): MediaWiki->performRequest()
#5 /var/www/live/w/includes/MediaWiki.php(527): MediaWiki->main()
#6 /var/www/live/w/index.php(44): MediaWiki->run()
#7 {main}

It appears to be possible to create new pages, however.

Event Timeline

This seems to only affect certain pages, specifically those edited between July and last week, as I was able to edit my user page and a template today.

lewiscawte renamed this task from Unable to edit existing pages to Botched Revision Data.Oct 20 2020, 8:39 PM
lewiscawte triaged this task as High priority.
lewiscawte added a subscriber: Saftzie.

Merged these in because they're all the same underlying cause which is some sort of botched revision (MediaWiki internals "recently" had a huge overhaul) which is causing it to freak itself out a little. We highly suspect the culprit is CreateWiki, however, at this time a cause and fix is proving difficult to pin down.

In T298#4466, @--miki-- wrote:

This seems to only affect certain pages, specifically those edited between July and last week, as I was able to edit my user page and a template today.

I previously got an error trying to edit the Main Page, although that was last edited in June just after the MW upgrade. I don't recall trying to edit other, older articles specifically. I can confirm that I can edit today.

Something I could do a few days ago, but cannot now, is even view the current rev of some articles.
Example: the Forum: article in T296

[1f6237ae275855e747cee44c] /wiki/Forum:I_don%27t_seem_to_be_an_admin_on_my_new_wiki
MediaWiki\Revision\RevisionAccessException from line 1442 of /var/www/live/w/includes/Revision/RevisionStore.php: Failed to load data blob from tt:159276: Unable to fetch blob at tt:159276

Backtrace:

#0 /var/www/live/w/includes/Revision/RevisionStore.php(1669): MediaWiki\Revision\RevisionStore->loadSlotContent(MediaWiki\Revision\SlotRecord, NULL, NULL, NULL, integer)
#1 [internal function]: MediaWiki\Revision\RevisionStore->MediaWiki\Revision\{closure}(MediaWiki\Revision\SlotRecord)
#2 /var/www/live/w/includes/Revision/SlotRecord.php(307): call_user_func(Closure, MediaWiki\Revision\SlotRecord)
#3 /var/www/live/w/includes/Revision/RevisionRecord.php(175): MediaWiki\Revision\SlotRecord->getContent()
#4 /var/www/live/w/includes/Revision/RenderedRevision.php(228): MediaWiki\Revision\RevisionRecord->getContent(string, integer, NULL)
#5 /var/www/live/w/includes/Revision/RevisionRenderer.php(215): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string)
#6 /var/www/live/w/includes/Revision/RevisionRenderer.php(152): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#7 [internal function]: MediaWiki\Revision\RevisionRenderer->MediaWiki\Revision\{closure}(MediaWiki\Revision\RenderedRevision, array)
#8 /var/www/live/w/includes/Revision/RenderedRevision.php(198): call_user_func(Closure, MediaWiki\Revision\RenderedRevision, array)
#9 /var/www/live/w/includes/poolcounter/PoolWorkArticleView.php(196): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#10 /var/www/live/w/includes/poolcounter/PoolCounterWork.php(125): PoolWorkArticleView->doWork()
#11 /var/www/live/w/includes/page/Article.php(791): PoolCounterWork->execute()
#12 /var/www/live/w/includes/actions/ViewAction.php(63): Article->view()
#13 /var/www/live/w/includes/MediaWiki.php(511): ViewAction->show()
#14 /var/www/live/w/includes/MediaWiki.php(302): MediaWiki->performAction(Article, Title)
#15 /var/www/live/w/includes/MediaWiki.php(854): MediaWiki->performRequest()
#16 /var/www/live/w/includes/MediaWiki.php(527): MediaWiki->main()
#17 /var/www/live/w/index.php(44): MediaWiki->run()
#18 {main}

So even the current revision of the article does not exist, although the article itself still does.

Is it time to lock the database yet? When was the last backup?

In T298#4477, @Saftzie wrote:

So even the current revision of the article does not exist, although the article itself still does.

Is it time to lock the database yet? When was the last backup?

I was tempted to merge this down to T296, because, ultimately, it is all the same cause afaik. I'm not entirely certain that is what the error is saying. I think it may well be looking in the wrong place for the revision, probably because some revisions from CW don't exist, although I've not had time to look into this as of yet - I've just had my first dissertation deadline which has absorbed pretty much all of my spare brain capacity.

Databases are backed up daily, but, I'm not reverting Hub. 90% of the Hub content is absolutely useless (there are... future... plans for it), and is no real loss to anyone if we have to make it vanish to fix this.

Backups can be used to restore lost data, not just revert. Revs should not be disappearing from the database. Nor should the problem be getting worse with time.

Further examples of pages that have disappeared due to database corruption:

User talk:Saftzie
User:Lcawte
User talk:Lcawte

I'd like my talk page restored, please, preferably with all the history.

Apparently the content still exists in DB because MW considered my edit to MediaWiki:Recentchanges-helppage null edit when I added exactly the same content it used to have. Now the content is shown on the page, but still not in the edit window. (and the message doesn't change the help link while it should)

Update: Root cause is tracked down, we think it's fixed (confirmation of that at a later point, currently isolated so it won't do any harm until then) and we've liberated as much data as we can from the void.

We'll be testing and then implementing the fix hopefully by the end of this week, but, the UK has just come out of another lockdown just before Christmas and my "day job" is in retail, so it is entirely dependent on me being able to keep my eyes open when I get home from a shift.

Please try and avoid "restoring" content yourselves, it makes playing around in the database to try and get accurate histories back even harder than it already is. I get the feeling revision history was never meant to be manually edited in SQL.