Okay, so just so we're clear, please hold off on proceeding with this until we've got our communications sorted. I'll post a note here when that's happened.

(In reply to comment #4)

Nonetheless, I strongly encourage an advance communication strategy (perhaps
using the community liaisons) to all Wikimedia projects where there are
oversighted edits. When the extension was in use, it was broadly and publicly
advertised as "permanent removal"

I feel that the burden for this communication should lie on the shoulders of the individuals who misled the community in this way.

Certainly, when we developed the OS tool (whilst I was on ArbCom), we were careful to note that the information was not actually removed from the database entirely, and that it was theoretically open to sub poena and viewing by people with special access, but that it was beyond the reach of mere administrators.

there is good reason to anticipate some negative reaction if there isn't
solid communication in advance.

Quite possibly. I'd suggest that the community 'functionaries' be alerted and asked to inform their communities. After all, they own the responsibility for keeping abreast of privileged rights; on enwiki this is ArbCom.

(In reply to comment #13)

(In reply to comment #4)
Nonetheless, I strongly encourage an advance
communication strategy (perhaps
using the community liaisons) to all
Wikimedia projects where there are
oversighted edits. When the extension
was in use, it was broadly and publicly
advertised as "permanent removal"
I feel that the burden for this communication should lie on the shoulders of
the individuals who misled the community in this way.

Well, I guess that would have been whomever set up the Oversight-L mailing list, which has always resulted in the following "TO" line:

"Requests to permanently remove personal information from the English Wikipedia <oversight-l@lists.wikimedia.org>"

It certainly said that in early 2007 (when I first requested an edit be oversighted).

There is a rather well-known case of oversight that occurred in 2007 where developers were requested to restore certain oversighted edits and refused (even though there was agreement the oversight was done in error), and thus the enwiki community has had reason to believe that there was no foreseeable circumstance under which an edit would be restored.

This is a really good opportunity for the WMF to have a positive communication interaction with the community - it's a net positive change, the responses to questions are positive and reassuring, and it will demonstrate the WMF's commitment to respecting user privacy (and telling them when something has changed in some way with respect to that) can effectively work hand-in-hand with software improvements. It's concerning that there's such vehement objection to communication, especially communication that is very likely to positively reflect good community/developer interaction.

[CC-ing some Commons oversighters so that they are aware of this change coming up. You're welcome.]

reedy@tin:/tmp$ grep count hidden.log | grep -v "] => 0"
akwiki: [count(*)] => 2
arwiki: [count(*)] => 1
arwikibooks: [count(*)] => 1
arwikinews: [count(*)] => 2
barwiki: [count(*)] => 1
bjnwiki: [count(*)] => 1
cebwiki: [count(*)] => 3
commonswiki: [count(*)] => 169
crwiki: [count(*)] => 2
cswiki: [count(*)] => 89
cywiki: [count(*)] => 1
dewiki: [count(*)] => 68
dewikibooks: [count(*)] => 16
dewikinews: [count(*)] => 27
dewiktionary: [count(*)] => 1
eewiki: [count(*)] => 4
enwiki: [count(*)] => 9905
enwikibooks: [count(*)] => 2
enwikinews: [count(*)] => 41
enwikiquote: [count(*)] => 16
enwikisource: [count(*)] => 6
enwikiversity: [count(*)] => 52
enwiktionary: [count(*)] => 17
eowiki: [count(*)] => 2
eswiki: [count(*)] => 62
eswikisource: [count(*)] => 1
eswiktionary: [count(*)] => 1
fawiki: [count(*)] => 10
fiwiki: [count(*)] => 113
foundationwiki: [count(*)] => 5
frwiki: [count(*)] => 130
frwikinews: [count(*)] => 2
frwikiquote: [count(*)] => 1
frwikisource: [count(*)] => 1
frwiktionary: [count(*)] => 1
gdwiktionary: [count(*)] => 1
glwiki: [count(*)] => 2
huwiki: [count(*)] => 1
idwiki: [count(*)] => 13
ikwiki: [count(*)] => 3
itwiki: [count(*)] => 54
jawiki: [count(*)] => 3
kowiki: [count(*)] => 1
lvwiki: [count(*)] => 5
mediawikiwiki: [count(*)] => 14
metawiki: [count(*)] => 125
mlwiki: [count(*)] => 1
mtwiki: [count(*)] => 1
nlwiki: [count(*)] => 81
nlwikimedia: [count(*)] => 1
pdcwiki: [count(*)] => 1
plwiki: [count(*)] => 21
ptwiki: [count(*)] => 125
ptwikisource: [count(*)] => 6
ruwiki: [count(*)] => 241
sewiki: [count(*)] => 6
simplewiki: [count(*)] => 26
slwiktionary: [count(*)] => 1
specieswiki: [count(*)] => 2
srwiki: [count(*)] => 7
stwiki: [count(*)] => 11
svwiki: [count(*)] => 462
tawiki: [count(*)] => 2
testwiki: [count(*)] => 12
wikimania2008wiki: [count(*)] => 18
wuuwiki: [count(*)] => 1
yiwiki: [count(*)] => 39
zeawiki: [count(*)] => 1
zhwiki: [count(*)] => 60
reedy@tin:/tmp$ grep count hidden.log | grep -v "] => 0" -c
69

(In reply to comment #13)

I feel that the burden for this communication should lie on the shoulders of
the individuals who misled the community in this way.

Agreed. Additionally, as the confused community luckily seems to be restricted to en.wiki, let's please not delay this long-overdue technical debt cleanup for all the others. Oversight has severe bugs.

I've reached out to the stewards, Commons oversighters, and English Wikipedia oversighters with an announcement about this.

Thanks for all of the answers so far. One more question: what will the log entries look like? Will they go into the suppression table and show the original oversighter and the original oversight log summary and date of oversight?

To note, I've created a FAQ for English Wikipedia, although I've noted a few questions that need to be verified. Others may feel free to translate for their wikis or otherwise copy.

https://en.wikipedia.org/wiki/User:Risker/Conversion_of_Oversighted_edits_to_Suppressed_edits

(In reply to comment #19)

Thanks for all of the answers so far. One more question: what will the log
entries look like? Will they go into the suppression table and show the
original oversighter and the original oversight log summary and date of
oversight?

Log entries will be added to Special:Log/suppress, and should look as if they had been created via the revdel system. Time, user, log summary will all be the same as the original oversight log entry.

(In reply to comment #21)

(In reply to comment #19)

Thanks for all of the answers so far. One more question: what will the log
entries look like? Will they go into the suppression table and show the
original oversighter and the original oversight log summary and date of
oversight?

Log entries will be added to Special:Log/suppress, and should look as if they
had been created via the revdel system. Time, user, log summary will all be
the
same as the original oversight log entry.

Thanks, Kunal. What will happen to the Oversight log? Will it remain intact or will it be deleted? Given that over 80% of all oversights (including a lot of weird edge cases) happened on enwiki, it would be useful for it to be kept intact, at least for a period, so that QA checks can be made. Not that there's likely to be a problem, simply that it would be impossible to tell if there is without the original logs for comparison.

Also, I'm assuming that this will fix the old attribution glitch. Is that correct?

(In reply to comment #22)

Thanks, Kunal. What will happen to the Oversight log? Will it remain intact
or will it be deleted? Given that over 80% of all oversights (including a
lot
of weird edge cases) happened on enwiki, it would be useful for it to be kept
intact, at least for a period, so that QA checks can be made. Not that
there's
likely to be a problem, simply that it would be impossible to tell if there
is
without the original logs for comparison.

The oversight log which is currently exposed via Special:Oversight will disappear whenever the extension is undeployed. As long as the hidden table isn't deleted (not sure what the WMF policy is on that), the log would still be recoverable by anyone with database access.

Also, I'm assuming that this will fix the old attribution glitch. Is that
correct?

I have no clue on what caused the glitch, so I don't know whether this will fix it.

(In reply to comment #23)

(In reply to comment #22)

Also, I'm assuming that this will fix the old attribution glitch. Is that
correct?

I have no clue on what caused the glitch, so I don't know whether this will
fix
it.

I've asked for longer-service oversighters who are probably more aware of the nature of this glitch to comment; I'll port over any responses I see in other venues. I only used this extension for a couple of months before we moved to suppression, so my knowledge is sketchy.

(In reply to comment #20)

[[User:Risker/Conversion of Oversighted edits to Suppressed edits]]

Thanks for this! :-) It probably makes sense to transwiki this page to Meta-Wiki and set it up for translation once it's fleshed out.

Thanks to all involved with this, it's a welcome fix to a legacy issue.

Are there any plans to change the name of the "oversighters" group as well following this transition?

(In reply to comment #26)

Thanks to all involved with this, it's a welcome fix to a legacy issue.

Are

there any plans to change the name of the "oversighters" group as well
following this transition?

Given that the logical change would be to "suppressors", I certainly hope not; it's even more Orwellian than "oversighters". Not to mention its similarity to "suppressive persons".

A discussion about the name change has been floating around for a few years, but there haven't been any updates to it since November 2011; see https://strategy.wikimedia.org/wiki/Proposal:Change_usergroup_name_%22Oversight%22 and bug 20337 for some details.

Even if there might be some plans to change the name, that's outside of Bugzilla's remit, since no software changes are necessary to change the on-wiki name of the group.

(In reply to comment #26)

Are there any plans to change the name of the "oversighters" group as well
following this transition?

There's no urgency for this, so we can consider this at a later date.

I think we're basically all set to go with the migration. I'll be asking for it to be carried out soon.

(In reply to comment #28)

Even if there might be some plans to change the name, that's outside of
Bugzilla's remit, since no software changes are necessary to change the
on-wiki name of the group.

Okay, thanks.

(In reply to comment #29)

(In reply to comment #26)
Are there any plans to change the name of the "oversighters" group as well
following this transition?

There's no urgency for this, so we can consider this at a later date.

Sure. I'm just thinking about how the documentation on enwp will need to be updated following this change. That's information enough to keep it current.

Okay Reedy, you can go ahead with this migration when you're ready. Please leave the oversight log intact (i.e. don't remove the old oversight extension yet).

Maybe we should get Gerrit change 111514 merged first.

(In reply to comment #32)

Maybe we should get Gerrit change #111514 merged first.

Yes please (just in case). Reedy ^

(In reply to comment #32)

Maybe we should get Gerrit change #111514 merged first.

Sounds sensible to me.

What is the status of this bug?

Still waiting on the latest gerrit change.

Okay, now that's done.

Chris and I are going to test this script tomorrow briefly. We can wait until then to run it on production.

Chris's tests were successful. Still, we can afford to take our time with this to make sure that it all works correctly in production. Here's the rollout plan for this:

Week 1: mediawiki.org
Week 2: wikiquotes, wikiversity, wikibooks, wikispecies, wikivoyage, wiktionary, wikinews, wikisource
Week 3: commons, meta, anything else I've forgotten except wikipedias
Week 4: wikipedias

We'll keep a week between deployments to see if there are any unexpected issues, and if there are then we'll delay future deployments. The oversight log will remain accessible after the migration so we can verify that everything went fine. I'm familiar with the operation of suppression and have global suppression by virtue of my staff rights, so I'll be available to help if anything goes wrong.

I'll write on here when I know what date this will begin.

Wikivoyages really shouldn't have anything oversight related in the dB...

(In reply to Sam Reed (reedy) from comment #40)

Wikivoyages really shouldn't have anything oversight related in the dB...

According to the script that was run above, they don't. I put it on there because I was using the site map to come up with my deployment list, and I kept miscounting when I'd left it out. ;-)

mysql says no

apache@tin:/tmp$ cat hidden.sql
select count(*) from hidden;

apache@tin:/tmp$ foreachwikiindblist /a/common/wikivoyage.dblist sql.php /tmp/hidden.sql

dewikivoyage

dewikivoyage: stdClass Object
dewikivoyage: (
dewikivoyage: [count(*)] => 0

dewikivoyage: )

elwikivoyage

elwikivoyage: stdClass Object
elwikivoyage: (
elwikivoyage: [count(*)] => 0

elwikivoyage: )

enwikivoyage

enwikivoyage: stdClass Object
enwikivoyage: (
enwikivoyage: [count(*)] => 0

enwikivoyage: )

eswikivoyage

eswikivoyage: stdClass Object
eswikivoyage: (
eswikivoyage: [count(*)] => 0

eswikivoyage: )

frwikivoyage

frwikivoyage: stdClass Object
frwikivoyage: (
frwikivoyage: [count(*)] => 0

frwikivoyage: )

hewikivoyage

hewikivoyage: stdClass Object
hewikivoyage: (
hewikivoyage: [count(*)] => 0

hewikivoyage: )

itwikivoyage

itwikivoyage: stdClass Object
itwikivoyage: (
itwikivoyage: [count(*)] => 0

itwikivoyage: )

nlwikivoyage

nlwikivoyage: stdClass Object
nlwikivoyage: (
nlwikivoyage: [count(*)] => 0

nlwikivoyage: )

plwikivoyage

plwikivoyage: stdClass Object
plwikivoyage: (
plwikivoyage: [count(*)] => 0

plwikivoyage: )

ptwikivoyage

ptwikivoyage: stdClass Object
ptwikivoyage: (
ptwikivoyage: [count(*)] => 0

ptwikivoyage: )

rowikivoyage

rowikivoyage: stdClass Object
rowikivoyage: (
rowikivoyage: [count(*)] => 0

rowikivoyage: )

ruwikivoyage

ruwikivoyage: stdClass Object
ruwikivoyage: (
ruwikivoyage: [count(*)] => 0

ruwikivoyage: )

svwikivoyage

svwikivoyage: stdClass Object
svwikivoyage: (
svwikivoyage: [count(*)] => 0

svwikivoyage: )

ukwikivoyage

ukwikivoyage: stdClass Object
ukwikivoyage: (
ukwikivoyage: [count(*)] => 0

ukwikivoyage: )

viwikivoyage

viwikivoyage: stdClass Object
viwikivoyage: (
viwikivoyage: [count(*)] => 0

viwikivoyage: )

zhwikivoyage

zhwikivoyage: stdClass Object
zhwikivoyage: (
zhwikivoyage: [count(*)] => 0
zhwikivoyage: )
apache@tin:/tmp$

(In reply to Dan Garry from comment #39)

Chris's tests were successful. Still, we can afford to take our time with
this to make sure that it all works correctly in production. Here's the
rollout plan for this:

Week 1: mediawiki.org
Week 2: wikiquotes, wikiversity, wikibooks, wikispecies, wikivoyage,
wiktionary, wikinews, wikisource
Week 3: commons, meta, anything else I've forgotten except wikipedias
Week 4: wikipedias

We'll keep a week between deployments to see if there are any unexpected
issues, and if there are then we'll delay future deployments. The oversight
log will remain accessible after the migration so we can verify that
everything went fine. I'm familiar with the operation of suppression and
have global suppression by virtue of my staff rights, so I'll be available
to help if anything goes wrong.

I'll write on here when I know what date this will begin.

Good plan, Dan. If I may suggest:

Week 4: all Wikipedias except enwiki

Week 5: Enwiki - See note below

Suggested QA reviews:

10 oversights - all

10>50 oversights - 10 reviews
50>100 oversights -20%
100+ oversights -10 to 15%
Enwiki - 5 to 10%

Suggest enlisting several enwiki oversighters to participate in the QA sample. Enwiki represents over 80% of oversighted edits, and if something is going to be weird, it will probably be here; when I look back at the logs, I see a lot of really strange actions.

I've done up a google doc[1] stratifying the proposed schedule, with suggested number of QA checks, based on the results Reedy got in Comment 16.

[1] https://docs.google.com/spreadsheet/ccc?key=0Ahh1Kb4u4Ot9dDFid3VSWF9pa1A4RDdsMWkwZ1A0eGc&usp=sharing

Good plan, Dan. You just need to execute it.

Did anything ever happen with this?

No.

I did mediawikiwiki ages ago... And I've just done everywhere else bar enwiki

reedy@tin:~$ grep "Done" oversight.log
akwiki: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
arwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
arwikibooks: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
arwikinews: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
barwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
bjnwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
cebwiki: Done! 3 oversighted revision(s) are now converted to suppressed revisions.
commonswiki: Done! 169 oversighted revision(s) are now converted to suppressed revisions.
crwiki: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
cswiki: Done! 89 oversighted revision(s) are now converted to suppressed revisions.
cywiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
dewiki: Done! 68 oversighted revision(s) are now converted to suppressed revisions.
dewikibooks: Done! 16 oversighted revision(s) are now converted to suppressed revisions.
dewikinews: Done! 27 oversighted revision(s) are now converted to suppressed revisions.
dewiktionary: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
eewiki: Done! 4 oversighted revision(s) are now converted to suppressed revisions.
enwikibooks: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
enwikinews: Done! 41 oversighted revision(s) are now converted to suppressed revisions.
enwikiquote: Done! 16 oversighted revision(s) are now converted to suppressed revisions.
enwikisource: Done! 6 oversighted revision(s) are now converted to suppressed revisions.
enwikiversity: Done! 52 oversighted revision(s) are now converted to suppressed revisions.
enwiktionary: Done! 17 oversighted revision(s) are now converted to suppressed revisions.
eowiki: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
eswiki: Done! 62 oversighted revision(s) are now converted to suppressed revisions.
eswikisource: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
eswiktionary: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
fawiki: Done! 10 oversighted revision(s) are now converted to suppressed revisions.
fiwiki: Done! 113 oversighted revision(s) are now converted to suppressed revisions.
foundationwiki: Done! 5 oversighted revision(s) are now converted to suppressed revisions.
frwiki: Done! 130 oversighted revision(s) are now converted to suppressed revisions.
frwikinews: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
frwikiquote: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
frwikisource: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
frwiktionary: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
gdwiktionary: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
glwiki: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
huwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
idwiki: Done! 13 oversighted revision(s) are now converted to suppressed revisions.
ikwiki: Done! 3 oversighted revision(s) are now converted to suppressed revisions.
itwiki: Done! 54 oversighted revision(s) are now converted to suppressed revisions.
jawiki: Done! 3 oversighted revision(s) are now converted to suppressed revisions.
kowiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
lvwiki: Done! 5 oversighted revision(s) are now converted to suppressed revisions.
metawiki: Done! 125 oversighted revision(s) are now converted to suppressed revisions.
mlwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
mtwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
nlwiki: Done! 81 oversighted revision(s) are now converted to suppressed revisions.
nlwikimedia: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
pdcwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
plwiki: Done! 21 oversighted revision(s) are now converted to suppressed revisions.
ptwiki: Done! 125 oversighted revision(s) are now converted to suppressed revisions.
ptwikisource: Done! 6 oversighted revision(s) are now converted to suppressed revisions.
ruwiki: Done! 241 oversighted revision(s) are now converted to suppressed revisions.
sewiki: Done! 6 oversighted revision(s) are now converted to suppressed revisions.
simplewiki: Done! 26 oversighted revision(s) are now converted to suppressed revisions.
slwiktionary: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
specieswiki: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
srwiki: Done! 7 oversighted revision(s) are now converted to suppressed revisions.
stwiki: Done! 11 oversighted revision(s) are now converted to suppressed revisions.
svwiki: Done! 462 oversighted revision(s) are now converted to suppressed revisions.
tawiki: Done! 2 oversighted revision(s) are now converted to suppressed revisions.
testwiki: Done! 12 oversighted revision(s) are now converted to suppressed revisions.
wikimania2008wiki: Done! 0 oversighted revision(s) are now converted to suppressed revisions.
wuuwiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
yiwiki: Done! 39 oversighted revision(s) are now converted to suppressed revisions.
zeawiki: Done! 1 oversighted revision(s) are now converted to suppressed revisions.
zhwiki: Done! 60 oversighted revision(s) are now converted to suppressed revisions.
reedy@tin:~$

Change 169611 had a related patch set uploaded by Reedy:
Only enable Extension:Oversight on enwiki

https://gerrit.wikimedia.org/r/169611

Change 169612 had a related patch set uploaded by Reedy:
Disable Extension:Oversight

https://gerrit.wikimedia.org/r/169612

Change 169611 merged by jenkins-bot:
Only enable Extension:Oversight on enwiki

https://gerrit.wikimedia.org/r/169611

@Reedy - you might have converted the oversighted edits to suppression (thanks for that), but you didn't convert/import the log actions. Which means we now have fully suppressed edits without any reasoning about why something was removed. See for example: https://nl.wikipedia.org/w/index.php?title=Speciaal:VersieVerwijderen&type=revision&target=Ad_van_den_Berg_%28activist%29&ids=23646821 (I remembered this one as it was the last oversighted edit on nlwiki). I assumed the log entries would be converted/imported too, but apparently not... (even a short note like "this edit was suppressed by xxx on xxx because of xxx via the now obsolete oversight extension" or something along those lines)

Well the script definitely has code to handle adding the log entries... Not sure why it didn't get run in production. Although obviously I can't investigate that.

Hello? Any update on this? @Reedy? The log actions are still not added...

@Krenair I guess it's worth testing it locally again to make sure it's doing what was expected...

Also, looking at more entries would be beneficial. If necessary, we can re-enable the Oversight extension more widely again to help find edits

I tested it locally and a Special:Log/suppress entry appeared with the correct details (basically the timestamp, user and reason that was given when it was Oversighted).

@Reedy should be able to check the wikipedia database for what happened to the log entry in the case you mention, @Trijnstel.

@Krenair I checked it again and it's kinda weird.... I searched in the suppression log on nlwiki and actually, there *is* an entry in the log (look for the date "19 dec 2010 15:04"), BUT... the log entry isn't mentioned on the suppression log of the suppressed edit as can be seen here: https://nl.wikipedia.org/w/index.php?title=Speciaal:VersieVerwijderen&type=revision&target=Ad_van_den_Berg_%28activist%29&ids=23646821 (if you know what I mean) And that should be fixed ofc.

Looks like it's expecting entries in the log_search table, which weren't added.

Change 169612 had a related patch set uploaded (by Alex Monk):
Disable Extension:Oversight

https://gerrit.wikimedia.org/r/169612

Change 224298 had a related patch set uploaded (by Alex Monk):
No longer branch Extension:Oversight

https://gerrit.wikimedia.org/r/224298

Hold on - have the oversighted edits on English Wikipedia been converted yet? Please don't disable until that has been done, we do periodically have to review some of that information.

In T62373#952148, @Trijnstel wrote:

@Krenair I checked it again and it's kinda weird.... I searched in the suppression log on nlwiki and actually, there *is* an entry in the log (look for the date "19 dec 2010 15:04"), BUT... the log entry isn't mentioned on the suppression log of the suppressed edit as can be seen here: https://nl.wikipedia.org/w/index.php?title=Speciaal:VersieVerwijderen&type=revision&target=Ad_van_den_Berg_%28activist%29&ids=23646821 (if you know what I mean) And that should be fixed ofc.

Has this been fixed?

In T62373#1447842, @Risker wrote:

Hold on - have the oversighted edits on English Wikipedia been converted yet? Please don't disable until that has been done, we do periodically have to review some of that information.

No, I'm planning to run it soon to bring enwiki up to speed with the other wikis. Then I'm going to do the above patches.

In T62373#1447843, @Risker wrote:

In T62373#952148, @Trijnstel wrote:

@Krenair I checked it again and it's kinda weird.... I searched in the suppression log on nlwiki and actually, there *is* an entry in the log (look for the date "19 dec 2010 15:04"), BUT... the log entry isn't mentioned on the suppression log of the suppressed edit as can be seen here: https://nl.wikipedia.org/w/index.php?title=Speciaal:VersieVerwijderen&type=revision&target=Ad_van_den_Berg_%28activist%29&ids=23646821 (if you know what I mean) And that should be fixed ofc.

Has this been fixed?

Not yet.

@Krenair - Enwiki has over 80% of all edits that were ever oversighted, around 10K as I recall. As noted, we do look at them from time to time. It would be helpful to have the problems identified on other wikis fixed and the fixes confirmed to be effective before running it on enwiki. (In fact, I thought the purpose of running it on the smaller wikis first was specifically to identify and fix problems before it was done on enwiki, where a much bigger mess could result.)

Change 224319 had a related patch set uploaded (by Alex Monk):
Add maintenance script to fix log_search of revisions migrated to revdel

https://gerrit.wikimedia.org/r/224319

Change 224319 merged by jenkins-bot:
Add maintenance script to fix log_search of revisions migrated to revdel

https://gerrit.wikimedia.org/r/224319

@Trijnstel, the script has been run on slwiktionary. Please check for logs on https://sl.wiktionary.org/wiki/Special:RevisionDelete?type=revision&ids=89131
If that works we can run it on nlwiki to check the given example, and if that's fine as well then I'll run it on all wikis other than enwiki (since the main migration script hasn't been run there yet, it'll get done at the same time).

Change 225504 had a related patch set uploaded (by Alex Monk):
Add an option to only migrate a maximum number of revisions at a time

https://gerrit.wikimedia.org/r/225504

Change 225504 merged by jenkins-bot:
Add an option to only migrate a maximum number of revisions at a time

https://gerrit.wikimedia.org/r/225504

In T62373#1455021, @Krenair wrote:

@Trijnstel, the script has been run on slwiktionary. Please check for logs on https://sl.wiktionary.org/wiki/Special:RevisionDelete?type=revision&ids=89131
If that works we can run it on nlwiki to check the given example, and if that's fine as well then I'll run it on all wikis other than enwiki (since the main migration script hasn't been run there yet, it'll get done at the same time).

Looks like it works (as I see the suppression log for that diff on slwiktionary), so I guess you can continue with nlwiki (and after that with the other wikis). Thanks for helping!

In T62373#1468884, @Trijnstel wrote:

In T62373#1455021, @Krenair wrote:

@Trijnstel, the script has been run on slwiktionary. Please check for logs on https://sl.wiktionary.org/wiki/Special:RevisionDelete?type=revision&ids=89131
If that works we can run it on nlwiki to check the given example, and if that's fine as well then I'll run it on all wikis other than enwiki (since the main migration script hasn't been run there yet, it'll get done at the same time).

Looks like it works (as I see the suppression log for that diff on slwiktionary), so I guess you can continue with nlwiki (and after that with the other wikis). Thanks for helping!

Done on nlwiki. Please check your given example https://nl.wikipedia.org/wiki/Special:RevisionDelete?type=revision&ids=23646821 - and any others you know of on that wiki.

In T62373#1472440, @Krenair wrote:

Done on nlwiki. Please check your given example https://nl.wikipedia.org/wiki/Special:RevisionDelete?type=revision&ids=23646821 - and any others you know of on that wiki.

@Trijnstel, did you have the opportunity to look at the above linked edit and make sure it was ok?

I have now. 😅 But it looks like it's not working yet.... Nothing in the suppression log entry on that diff (and for that diff) yet...

Ah: https://nl.wikipedia.org/w/index.php?title=Ad_van_den_Berg_(politicus)&action=history
The recorded suppression log entries are against the old title. And indeed when you suppress a revision and then someone later moves the page to a new title, the suppression logs are left behind. So this is consistent with normal behaviour.

I checked the full SQL query it generates, and all seems fine otherwise. I guess we need to find another suppressed revision (which hasn't since been moved) to test this against.

We found other pages which haven't since been moved and the logs on Special:RevisionDelete worked fine.
Next steps:

• Run log_search fix script everywhere that isn't enwiki (no OS migration run there yet), slwiktionary (already done), nlwiki (already done)
• Run OS migration script on enwiki, then log_search fix script.

Failures on these wikis which ended up going into exception.log:
azbwiki
cnwikimedia
gomwiki
lrcwiki

(None of these wikis were created with the Oversight extension tables, so this is expected. I just forgot to remove them from my dblist)

And so that just leaves:

Run OS migration script on enwiki, then log_search fix script.

Change 231898 had a related patch set uploaded (by Alex Monk):
Migration script: Set last revision ID when ignoring a row

https://gerrit.wikimedia.org/r/231898

Change 231898 merged by jenkins-bot:
Migration script: Set last revision ID when ignoring a row

https://gerrit.wikimedia.org/r/231898

Note that enwiki contained a few entries where hidden_rev_id IS NULL, these cannot be fixed.

Change 169612 merged by jenkins-bot:
Disable Extension:Oversight

https://gerrit.wikimedia.org/r/169612