Drop support for Vega 1.5
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	VulpesVulpes825
	Aug 17 2020, 1:32 PM

Description

Eventually, we will want to start migrating to a latest version of Vega. Before we complete that, we need to deprecate and drop support for Vega 1.5 so we will not have a huge technical debt of supporting three Vega version (1.5, 2.6, 5.x).

The goal of this task is to flush out any last remaining usage of Vega 1.5 across the Wikimedia projects, migrate them to 2.6 definitions and then remove support for Vega 1.5.

announce in tech news this will happen
make initial measure of the prevalence of remaining usage of Vega 1.5 and identify wiki and templates needing support for migration.
flush out straggles by logging client side errors
drop Vega 1.5 and mop up.

How help to fix old syntax?

@Seddon collects problematic pages lower on this page
You can also use insource: "graph version 1" in search across all pages in your project or check this category - https://www.wikidata.org/wiki/Q21705175
Decide if you need this page and if it makes sense to update it
Most often it is enough to remove data. in field: https://pl.wikipedia.org/w/index.php?title=Sejm_Rzeczypospolitej_Polskiej_VII_kadencji&diff=prev&oldid=69935436
Or you can replace with modules: https://ru.wikipedia.org/w/index.php?diff=129411182&oldid=76542710
More info here: https://www.mediawiki.org/wiki/Extension:Graph/Vega_2.0
More info here too: https://web.archive.org/web/20160201003310/https://github.com/vega/vega/wiki/Upgrading-to-2.0
Old Vega 2 editor with examples: http://vega.github.io/vega-editor/?mode=vega

Details

	Subject	Repo	Branch	Lines +/-
	Remove Vega 1 and Vega 2 libraries from extension	mediawiki/extensions/Graph	master	+82 -34 K
	Remove vega1 from Graph extension	mediawiki/extensions/Graph	master	+22 -11 K

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Open	BUG REPORT	CCiufo-WMF	T334940 All Graphs broken on Wikimedia wikis (due to security issue T336556)
Resolved	Security	Jdlrobson	T334895 XSS via Graph extension
Duplicate		None	T292341 [Epic] Graphs extension maintenance
Declined		None	T335128 Update documentation and examples of Extension:Graph after deployment of Vega 5
Declined		None	T346291 Re-enable the Graph Extension for use at all Wikimedia Wikis
Declined	Feature	Jdlrobson	T165118 Support Vega 5.0+
Resolved		Jdlrobson	T260542 Drop support for Vega 1.5

Event Timeline

VulpesVulpes825 created this task.Aug 17 2020, 1:32 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 17 2020, 1:32 PM

VulpesVulpes825 renamed this task from Drop support for Vega 1.0 to Drop support for Vega 1.5.Aug 17 2020, 4:31 PM

VulpesVulpes825 updated the task description. (Show Details)

Aklapper awarded a token.Aug 17 2020, 4:31 PM

VulpesVulpes825 updated the task description. (Show Details)Aug 17 2020, 4:32 PM

TheDJ awarded a token.Aug 18 2020, 12:42 PM

Seddon updated the task description. (Show Details)Oct 2 2021, 12:24 AM

Seddon added a parent task: T292341: [Epic] Graphs extension maintenance.Oct 3 2021, 12:04 AM

SELECT uri_host, COUNT(*) FROM wmf.webrequest
WHERE year = 2021 AND month = 9 AND day = 10
AND uri_path = '/w/load.php'
AND uri_query LIKE '%vega1%'
GROUP BY uri_host

A typical days data

fr.wikipedia.org	99
pl.wikipedia.org	56
it.wikipedia.org	14
no.wikipedia.org	3
fr.wikibooks.org	1
ru.wikipedia.org	1
meta.wikimedia.org	1
es.wikipedia.org	1

Other sporadic instances from other days

my.wikipedia.org

www.mediawiki.org

en.wikipedia.org

fr.wiktionary.org

outreach.wikimedia.org

he.wikipedia.org

fr.wikipedia.org

https://fr.wikipedia.org/wiki/Mod%C3%A8le:Histogramme_%C3%A0_4_classes_pour_sp%C3%A9l%C3%A9om%C3%A9trie_d%C3%A9partementale

pl.wikimedia.org

it.wikipedia.org

no.wikipedia.org

https://no.wikipedia.org/wiki/Arbeidsledighet

It seems that the remaining Vega 1.5 usages are simple bar graphs, which should be easily upgraded by using the Module:Graph or manual updates.

An easier solution is to generate an error message when version number is 1 and ask editors for upgrades, considering how low the usage of Vega 1.5 is at this point.

@Seddon can you get updated numbers ?

Change 755513 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/Graph@master] Remove vega1 from Graph extension

https://gerrit.wikimedia.org/r/755513

gerritbot added a project: Patch-For-Review.Jan 20 2022, 12:27 AM

Base subscribed.Jan 20 2022, 12:52 AM

Change 755513 abandoned by TheDJ:

[mediawiki/extensions/Graph@master] Remove vega1 from Graph extension

Reason:

https://gerrit.wikimedia.org/r/755513

Maintenance_bot removed a project: Patch-For-Review.Oct 29 2022, 11:30 PM

Tech/News Proposed Copy
Vega 1.5 support is planned to be disabled in the near future, so some charts may break.

@Seddon can you update info about this change, please? :)

Quiddity moved this task from To Triage to Not ready to announce on the User-notice board.Mar 23 2023, 10:31 PM

Thanks!

In T260542#8723570, @Seddon wrote:

Pages requiring fixing:
ruwiki

[] https://ru.wikipedia.org/wiki/%D0%A3%D1%87%D0%B0%D1%81%D1%82%D0%BD%D0%B8%D0%BA:%D0%90%D0%BB%D0%B5%D0%BA%D1%81%D0%B0%D0%BD%D0%B4%D1%80%D0%9F346545
[] https://ru.wikipedia.org/wiki/%D0%92%D0%B8%D0%BA%D0%B8%D0%BF%D0%B5%D0%B4%D0%B8%D1%8F:%D0%A4%D0%BE%D1%80%D1%83%D0%BC/%D0%90%D1%80%D1%85%D0%B8%D0%B2/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%D0%B9/2015/06

Fixed for ruwiki :)

Hiyo. For Tech News, I'm not entirely sure what to write...

From the comments above, it looks like you've found and fixed ALL the existing instances already - is that accurate?

Additionally, it's not currently clear to me what editors need to look for in order to detect a problem, and if they find any, then how they should best fix it.
E.g. I looked at some of the pages linked above, and found these 2 examples in the page-histories:

Is one of those more preferable than the other? (Maybe they're only different because one is via a template? Maybe they're both equally-future-proof for the "[version] 5.13" mentioned in the Task Description?)

I also found this old page, which might be relevant(?): https://www.mediawiki.org/wiki/Extension:Graph/Vega_2.0

Ideally, readers need to understand the context, the problem, and how they can help.
For Tech News, something like this might work? (if accurate?!)

The Vega software that creates data visualizations in pages, such as graphs, will be upgraded to version 5 in the future. Pages that still use the very old version 1.0/1.5 syntax may stop working properly. Most existing uses have been found and updated, but you can help to check, and to update any local documentation. Examples of how to find and fix these pages are available.

That last sentence could be a link to this task?
Plus, someone could add a very short section at the top of the Task Description here, giving:

links to example-searches (maybe an insource:... search?)
links to example-fixes (so that they have a quick-reference), and
possibly one or more documentation pages (anything specific from within https://www.mediawiki.org/wiki/Extension:Graph#Useful_links ?).

TLDR: Please help to draft a clear message for Tech News, and to update the Task Description with any extra details needed. Thank you!
(I'll postpone the Tech News entry until the following week's edition, so we have ~6 days to get it done clearly. :)

P.s. Someone mentioned these pages, which might also be useful to include above?
https://de.wikipedia.org/wiki/Modul:Graph
https://de.wikipedia.org/wiki/Wikipedia:Lua/Modul/Graph/en

In T260542#8725754, @Quiddity wrote:

Hiyo. For Tech News, I'm not entirely sure what to write...

From the comments above, it looks like you've found and fixed ALL the existing instances already - is that accurate?

Additionally, it's not currently clear to me what editors need to look for in order to detect a problem, and if they find any, then how they should best fix it.
E.g. I looked at some of the pages linked above, and found these 2 examples in the page-histories:

https://pl.wikipedia.org/w/index.php?title=Sejm_Rzeczypospolitej_Polskiej_VII_kadencji&diff=prev&oldid=69935436

https://ru.wikipedia.org/w/index.php?title=%D0%A3%D1%87%D0%B0%D1%81%D1%82%D0%BD%D0%B8%D0%BA%3A%D0%90%D0%BB%D0%B5%D0%BA%D1%81%D0%B0%D0%BD%D0%B4%D1%80%D0%9F346545&diff=129411182&oldid=76542710

Is one of those more preferable than the other? (Maybe they're only different because one is via a template? Maybe they're both equally-future-proof for the "[version] 5.13" mentioned in the Task Description?)

I also found this old page, which might be relevant(?): https://www.mediawiki.org/wiki/Extension:Graph/Vega_2.0

Ideally, readers need to understand the context, the problem, and how they can help.
For Tech News, something like this might work? (if accurate?!)

The Vega software that creates data visualizations in pages, such as graphs, will be upgraded to version 5 in the future. Pages that still use the very old version 1.0/1.5 syntax may stop working properly. Most existing uses have been found and updated, but you can help to check, and to update any local documentation. Examples of how to find and fix these pages are available.

That last sentence could be a link to this task?
Plus, someone could add a very short section at the top of the Task Description here, giving:

links to example-searches (maybe an insource:... search?)

links to example-fixes (so that they have a quick-reference), and

possibly one or more documentation pages (anything specific from within https://www.mediawiki.org/wiki/Extension:Graph#Useful_links ?).

TLDR: Please help to draft a clear message for Tech News, and to update the Task Description with any extra details needed. Thank you!
(I'll postpone the Tech News entry until the following week's edition, so we have ~6 days to get it done clearly. :)

Hello! Thanks for such a detailed explanation! I'll think about it and come back if no one writes before.

Fix targets from March 27

Seddon added a comment.Mar 29 2023, 3:00 AM

This comment was removed by Seddon.

Iniquity added a project: Tech Ambassadors & Translators.Mar 29 2023, 11:03 AM

Iniquity updated the task description. (Show Details)Mar 29 2023, 11:23 AM

@Quiddity I like your version of note, but with some fixes:

The Vega software that creates data visualizations in pages, such as graphs, will be upgraded to newest version in the future. Graphs that still use the very old version 1.5 syntax may stop working properly. Most existing uses have been found and updated, but you can help to check, and to update any local documentation. Examples of how to find and fix these graphs are available.

Iniquity updated the task description. (Show Details)Mar 29 2023, 12:34 PM

Iniquity updated the task description. (Show Details)Mar 29 2023, 12:43 PM

Restricted Application added a project: Internet-Archive. · View Herald TranscriptMar 29 2023, 12:43 PM

Quiddity moved this task from Announce in next Tech/News to In current Tech/News draft on the User-notice board.Mar 31 2023, 12:05 AM

Frostly moved this task from In current Tech/News draft to Already announced/Archive on the User-notice board.Apr 4 2023, 2:25 AM

Frostly updated the task description. (Show Details)

Pamputt subscribed.Apr 4 2023, 7:03 AM

Iniquity moved this task from Backlog to Vega Update Phase 1 on the MediaWiki-extensions-Graph board.Apr 4 2023, 6:56 PM

Iniquity mentioned this in T165118: Support Vega 5.0+.Apr 7 2023, 1:33 PM

Thanks @Iniquity! Starting from next week I will start running the query daily.

If we can get through a week or two weeks without any instances in the web requests I think we can start feeling more confident about moving forward with the deprecation.

Iniquity edited parent tasks, added: T165118: Support Vega 5.0+; removed: T292341: [Epic] Graphs extension maintenance.Apr 7 2023, 3:19 PM

Iniquity mentioned this in T292341: [Epic] Graphs extension maintenance.Apr 7 2023, 3:29 PM

Iniquity updated the task description. (Show Details)Apr 7 2023, 4:23 PM

In T260542#8765110, @Seddon wrote:

Thanks @Iniquity! Starting from next week I will start running the query daily.

If we can get through a week or two weeks without any instances in the web requests I think we can start feeling more confident about moving forward with the deprecation.

Hi :) Have you run the query?

There's a problem with rendering graphs on plwiki recently. Many graphs don't show at all, for example in: https://pl.wikipedia.org/wiki/Ren_(pierwiastek)#Rynek (the graph should be generated by {{Wykres}} after the second paragraph in that section). There hasn't been any edits to the graph template and module. Is it related to this task?

It seems that at first the <graph> tag was emitted directly to the page content and purging results in just nothing in place of the graph. The pre-purge page content:

@Msz2001, graphs are disabled at the moment following a security issue tracked in currently non-public T334895

Thanks!

Created a separate task: T334940: All Graphs broken on Wikimedia wikis (due to security issue T336556).

Change 910060 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[mediawiki/extensions/Graph@master] Remove Vega 1 and Vega 2 libraries from extension

https://gerrit.wikimedia.org/r/910060

gerritbot added a project: Patch-For-Review.Apr 19 2023, 6:58 PM

Jdlrobson claimed this task.Apr 20 2023, 7:39 PM

Change 910060 merged by jenkins-bot:

[mediawiki/extensions/Graph@master] Remove Vega 1 and Vega 2 libraries from extension

https://gerrit.wikimedia.org/r/910060

Maintenance_bot removed a project: Patch-For-Review.Apr 24 2023, 7:10 AM

The security issue forced a decision here. The old versions have been removed from the Graph extension so support has effectively been dropped. We've also added error logging in place to catch graphs that are written in older schemas.

In T335048 we are exploring ways to help community update graphs more efficiently. I will let the error logging drive whether we do anything further for Vega 1.5. Please follow T335048 for further information regarding that.
In T335127 we will make sure we have any tagging for graphs using outdated schemas to help locate any problems.

I really hope that is sufficient please let me know if not.

Maintenance_bot edited projects, added User-notice-archive; removed User-notice.May 4 2023, 11:30 PM

ReleaseTaggerBot added a project: MW-1.41-notes (1.41.0-wmf.9; 2023-05-15).May 15 2023, 7:01 PM

Umherirrender mentioned this in T277903: RangeError: Invalid array length / TypeError: datum.firstYear is undefined / Uncaught TypeError: Cannot read property 'value' of undefined / SyntaxError: Failed to execute 'addColorStop' on 'CanvasGradient': The value provided ('#NaNNaNNaN') could not be parsed as a color.Jun 3 2023, 9:26 PM

Umherirrender mentioned this in T277907: [logspam, iPhone] TypeError: undefined is not an object (evaluating 'scene.marktype').

Umherirrender mentioned this in T293786: TypeError: Cannot read properties of undefined (reading 'marktype').

Umherirrender mentioned this in T258170: Uncaught TypeError: Cannot read property 'marktype' of undefined in CanvasHandler.pick() / undefined.