1081712 - Upgrade Semantic extensions

Status: RESOLVED FIXED

(Websites :: wiki.mozilla.org, defect)

Description

[Gordon P. Hemsley [:GPHemsley]]

A lot of our existing sec bugs are due to problems in the Semantic extensions. However, we don't appear to be running the latest versions of them.

* Semantic MediaWiki
  https://semantic-mediawiki.org/
* Semantic Forms
  https://www.mediawiki.org/wiki/Extension:Semantic_Forms
* Semantic Watchlist
  https://www.mediawiki.org/wiki/Extension:Semantic_Watchlist
* SMWAskAPI
  http://sourceforge.net/projects/smwaskapi/

It's possible that upgrading them would eliminate some of our sec bugs.

Comment 1

[Gordon P. Hemsley [:GPHemsley]]

This is something webops has to do.

Comment 2

[Christie Koehler [:ckoehler]]

> * Semantic MediaWiki
>   https://semantic-mediawiki.org/

This should be upgraded to 2.0:
https://packagist.org/packages/mediawiki/semantic-media-wiki#2.0

> * Semantic Forms
>   https://www.mediawiki.org/wiki/Extension:Semantic_Forms

Upgrade to 2.7:
https://git.wikimedia.org/commit/mediawiki%2Fextensions%2FSemanticForms.git/8820cb92e4010200252be3fdaf3df00c76bdb719

> * Semantic Watchlist
>   https://www.mediawiki.org/wiki/Extension:Semantic_Watchlist

Upgrade to 0.2.2:
https://github.com/SemanticMediaWiki/SemanticWatchlist/releases/tag/0.2.2

> * SMWAskAPI
>   http://sourceforge.net/projects/smwaskapi/

We're already at the most recent version of this (0.9a), so no need to upgrade.

Comment 3

[Gordon P. Hemsley [:GPHemsley]]

Hey C, can you elaborate on what pushed this out?

Comment 4

[C. Liang [:cyliang]]

Too many things on the plate to get it done last week.  Doing an import of the DB into dev takes roughly 30 minutes and, during that time, we can't do anything *else* to dev.

Comment 5

[Jason Crowe [:jd]]

I have upgraded these in dev. We get these from:
https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Semantic*
and the version 2.0 for SemanticMediawiki is not there yet so I have set it to track the 1.9.x branch.

The versions are set as follows:
Semantic MediaWiki
1.9.x
Semantic_Forms
2.8
Semantic Watchlist
0.2.2

nb: SemanticMediawiki can not track the HEAD^ of REL1_23 brnach as it breaks the site.

Comment 6

[Gordon P. Hemsley [:GPHemsley]]

(In reply to C. Liang [:cyliang] from comment #4)
> Too many things on the plate to get it done last week.  Doing an import of
> the DB into dev takes roughly 30 minutes and, during that time, we can't do
> anything *else* to dev.

That's cool. I'm just an idiot and didn't notice you were pushing out from last week rather than from this week.

(In reply to Jason Crowe [:jd] from comment #5)
> nb: SemanticMediawiki can not track the HEAD^ of REL1_23 brnach as it breaks
> the site.

Can you elaborate as to why? By design, that's not supposed to happen.

Comment 7

[Jason Crowe [:jd]]

(In reply to Gordon P. Hemsley [:GPHemsley] from comment #6)
> > nb: SemanticMediawiki can not track the HEAD^ of REL1_23 brnach as it breaks
> > the site.
> 
> Can you elaborate as to why? By design, that's not supposed to happen.

Unfortunately I did not save the error message, however when I set it to track HEAD the site would not load and had an error. If you want the error I can change it to track HEAD and collect the actual error.

Let me know

Comment 8

[Gordon P. Hemsley [:GPHemsley]]

(In reply to Jason Crowe [:jd] from comment #7)
> (In reply to Gordon P. Hemsley [:GPHemsley] from comment #6)
> > > nb: SemanticMediawiki can not track the HEAD^ of REL1_23 brnach as it breaks
> > > the site.
> > 
> > Can you elaborate as to why? By design, that's not supposed to happen.
> 
> Unfortunately I did not save the error message, however when I set it to
> track HEAD the site would not load and had an error. If you want the error I
> can change it to track HEAD and collect the actual error.
> 
> Let me know

Sure, might as well. It's only dev. :)

Comment 9

[Christie Koehler [:ckoehler]]

(In reply to Jason Crowe [:jd] from comment #5)
> I have upgraded these in dev. We get these from:
> https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Semantic*
> and the version 2.0 for SemanticMediawiki is not there yet so I have set it
> to track the 1.9.x branch.

Why the source listed above for SemanticMediaWiki and not this one, which is listed by maintainers as canonical?
https://github.com/SemanticMediaWiki/SemanticMediaWiki/releases/tag/2.0

I can't quite tell what's supposed to be at the gerrit.wikimedia.org link since no results are listed.

Comment 10

[Jason Crowe [:jd]]

(In reply to Christie Koehler [:ckoehler] from comment #9)
> (In reply to Jason Crowe [:jd] from comment #5)
> > I have upgraded these in dev. We get these from:
> > https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Semantic*
> > and the version 2.0 for SemanticMediawiki is not there yet so I have set it
> > to track the 1.9.x branch.
> 
> Why the source listed above for SemanticMediaWiki and not this one, which is
> listed by maintainers as canonical?
> https://github.com/SemanticMediaWiki/SemanticMediaWiki/releases/tag/2.0
> 
> I can't quite tell what's supposed to be at the gerrit.wikimedia.org link
> since no results are listed.

Beats me. I am happy to switch them to something else if you like. As long as it is git based and web accessible. Just give me the URLs you want me to use and I will switch things around.

Comment 11

[Gordon P. Hemsley [:GPHemsley]]

(In reply to Jason Crowe [:jd] from comment #10)
> (In reply to Christie Koehler [:ckoehler] from comment #9)
> > (In reply to Jason Crowe [:jd] from comment #5)
> > > I have upgraded these in dev. We get these from:
> > > https://gerrit.wikimedia.org/r/p/mediawiki/extensions/Semantic*
> > > and the version 2.0 for SemanticMediawiki is not there yet so I have set it
> > > to track the 1.9.x branch.
> > 
> > Why the source listed above for SemanticMediaWiki and not this one, which is
> > listed by maintainers as canonical?
> > https://github.com/SemanticMediaWiki/SemanticMediaWiki/releases/tag/2.0
> > 
> > I can't quite tell what's supposed to be at the gerrit.wikimedia.org link
> > since no results are listed.
> 
> Beats me. I am happy to switch them to something else if you like. As long
> as it is git based and web accessible. Just give me the URLs you want me to
> use and I will switch things around.

Indeed, Gerrit is Wikimedia's code review tool. I don't think it is intended to be the canonical repository for anything.

This SHOULD be where it's safe to pull from:
https://git.wikimedia.org/log/mediawiki%2Fextensions%2FSemanticMediaWiki.git/refs%2Fheads%2FREL1_23

However, it seems SMW has decided to buck the trend and live outside the MediaWiki ecosystem. I think the GitHub link that Christie provided is the latest available. Nevertheless, proceed with caution.

Comment 12

[Jason Crowe [:jd]]

For reference, this query is necessary for this update:
ALTER TABLE swl_groups ADD COLUMN group_custom_texts BLOB NULL;

As noted here:
http://www.mediawiki.org/wiki/Extension:Semantic_Watchlist#Updating_from_version_0.1

Comment 13

[Gordon P. Hemsley [:GPHemsley]]

FWIW, I just discovered Semantic Bundle, which packages all these related extensions (and more?) together:

https://www.mediawiki.org/wiki/Semantic_Bundle

Regardless, we should push whatever we can through the pipeline ASAP. If we can't get SMW updated to the latest version, we should file a separate bug for it and follow up later.

Comment 14

[Jason Crowe [:jd]]

I pulled it in from the github location. I am hoping to get some time tomorrow to hopefully finish the dev site restructuring and once that works this will follow.

Comment 15

[Gordon P. Hemsley [:GPHemsley]]

(In reply to Jason Crowe [:jd] from comment #14)
> I pulled it in from the github location. I am hoping to get some time
> tomorrow to hopefully finish the dev site restructuring and once that works
> this will follow.

Good to know, especially because I just noticed that roughly half of our (known) security bugs will be fixed by these upgrades.

Comment 16

[Christie Koehler [:ckoehler]]

This is pending JDs re-work of deployment configuration. Rescheduling for next week.

Comment 17

[Jason Crowe [:jd]]

This is wrapped up in the new deployment model and is going out as part of that deployment process.

Dev is already updated. If all goes well stage will be updated tomorrow with prod to follow in one week.

Comment 18

[Jason Crowe [:jd]]

This has been deployed to production.