Wikidata talk:Creating a bot

Starting this page

Wikidata depends on bots, so I start this page and call other bot managers to share their tips and experience. Also, sharing source code under open-source licenses is a must (I use GPLv3, but MIT or other are nice too). Don't reinvent the wheel and share your knowledge! Emijrp (talk) 16:21, 17 March 2013 (UTC)[reply]

Hi

This is a very good idea! But could you also mention how to create a bot without using pywikipedia? For instance one can get the item from a wikipedia article name via

def get_qnumber(wikiarticle):
	url="http://www.wikidata.org/w/api.php?action=wbgetentities&titles="+wikiarticle+"&sites=dewiki&props=&format=jsonfm"
	req = urllib2.Request(url, headers={'User-Agent' : "Magic Browser"})
	u = urllib2.urlopen(req)
	#read url
	htmltext=u.read()
	#get id
	qnumber_str=re.findall(ur'\&quot\;id\&quot\;\:\s\&quot\;(q[0-9]+)\&quot\;',htmltext)
	return qnumber_str

meaning, using the API directly.

My questions: How can I add an item with the same method? Or how to edit labels/descriptions or how to add claims. I found some api-functions like wbsetclaimvalue but I don't really get how they work.--Svebert (talk) 09:17, 18 March 2013 (UTC)[reply]

Yes, a section for API queries would be nice. Your questions are solved with a POST query (getting and sending an edit token). I haven't tested it yet but I saw something like that in some scripts. I'm going to add your code but with some improvements. I hope more people send their examples! Regards. Emijrp (talk) 09:38, 18 March 2013 (UTC)[reply]

pagegeneretors.py

It would be very useful, if these scripts could handle parameters from pagegenerators.pym like -file or -cat(egory), so we can run bot e.g. for adding same description for all pages from one category. JAn Dudík (talk) 11:02, 18 March 2013 (UTC)[reply]

Sure. The examples go from the basics to advanced results. So, after the introduction examples we can start to create examples with pagegenerators. Emijrp (talk) 13:00, 18 March 2013 (UTC)[reply]

great idea, code listing?

Thanks for starting this much needed page. Would it make sense to start an index of open source wikidata bot projects and keep it up here? We have code https://bitbucket.org/sulab/pygenewiki that will be moving from wikipedia updates to wikidata updates this summer. Happy to share in the experiences and the code. Genewiki123 (talk)

Sure, start a section == Other projects == at the bottom. Add a brief description about the project purpose. Emijrp (talk) 20:51, 19 March 2013 (UTC)[reply]

newbee

Sorry, I'm just beginning, but after having logged in obviously successfully:

Should be logged in now

when I run the first example, I get:

Traceback (most recent call last):
  File "C:\Python27\Mes_exemples\exemple1.py", line 3, in <module>
    import wikipedia as pywikibot
ImportError: No module named wikipedia

Any idea?
--Gloumouth1 (talk) 14:55, 24 March 2013 (UTC)[reply]

I have reverted this. Try now. Emijrp (talk) 19:56, 24 March 2013 (UTC)[reply]

editclaim in Pywikipediabot

For add a claim I use

data.editclaim(property, value, refs={(ref_source,val_source)})

but if I must add more values to the same property? --ValterVB (talk) 10:34, 1 May 2013 (UTC)[reply]

I think that PWB already supports multi-value claims; however, I use PWB-rewrite, which offers separate methods: claim.setTarget and item.addClaim. --Ricordi samoa 13:54, 1 May 2013 (UTC)[reply]

Two calls to server for interwiki

Hello, am I right now we should call server twice to get interwiki. First we get the id of item and then load it. In the old scenario without wikidata, we needed only one call: load page. It will give impact on big number (100) of article to check for interwiki. In the old scenario we used 100 calls and now we will use 200 calls to server--Alex Blokha (talk) 18:32, 14 June 2013 (UTC).[reply]

Bots in other languages?

Sadly, I don't know Python. Is there a module/library/whatever to write a bot in Perl or Lua? —Scott5114 ↗ [EXACT CHANGE ONLY] 08:50, 2 July 2013 (UTC)[reply]

@Scott5114: just learn Python!

--Ricordi samoa 10:08, 17 April 2014 (UTC)[reply]

ArgumentOutOfRangeException in DotNetDataBot

In DotNetDataBot method GetIdBySitelink in object of Item type don't work. I received ArgumentOutOfRangeException with message "StartIndex value can not be less than zero".

 StackTrace:
      in System.String.InternalSubStringWithChecks(Int32 startIndex, Int32 length, Boolean fAlwaysCopy)
      in System.String.Substring(Int32 startIndex)
      in DotNetDataBot.Item.getAPIError(String result)
      in DotNetDataBot.Item.getId(String result)
      in DotNetDataBot.Item.GetIdBySitelink(String lang, String sitelink)

Can anyone fix a bug or tell me what the problem? --Dymitr (talk) 09:04, 14 September 2013 (UTC)[reply]

The problem is with this. To solve it You must edit Item.cs, Search function private int getId(string result) and replace string mark = " id=\"q"; with string mark = " id=\"Q"; --ValterVB (talk) 12:35, 14 September 2013 (UTC)[reply]

Thanks a lot. --Dymitr (talk) 12:22, 5 October 2013 (UTC)[reply]

login problem

Hello,

I have a strange problem with pywikipediabot

my user-config.py :

mylang='wikidata'
family = 'wikidata'
usernames['wikidata']['wikidata']=u'escabot'
console_encoding = 'utf-8'

I run login.py and type my password , that's seem ok since I get "Should be logged in now"
Then I run my script which does something like :

repo = pywikibot.getSite('wikidata', 'wikidata')
data = pywikibot.DataPage(repo, qcode) 
data.editclaim(pcode, value)

The edit is ok but is done by under my IP instead of using escabot account.

Any help is welcome,

Ske (talk) 12:25, 15 September 2013 (UTC)[reply]

It might be "Escabot" instead of "escabot". Otherwise, you might want to try:

that was the problem, thank's Ske (talk) 13:46, 15 September 2013 (UTC)[reply]

family = 'wikipedia'
mylang = 'en'
usernames['wikidata']['wikidata'] = u'escabot'
usernames['wikipedia']['en'] = u'escabot'

Cheers. -- Docu at 12:44, 15 September 2013 (UTC)[reply]

--ValterVB (talk) 18:38, 24 September 2013 (UTC)[reply]

API request error

Someone know what is wrong in this API request used on Wikidata Sandbox (Q4115189)?

summary=summary&baserevid=72068527&token=5d6e2dbc4876ce1947f5d55d9145ab6f+\&statement=Q4115189$c21ed08d-43d5-57e8-89da-1495dad4b4b3&snaks={"P370":[{"snaktype":"value","property":"P370","datavalue":{"type":"string","value":"Testo"}}}

Return

<?xml version="1.0"?><api servedby="mw1135"><error code="invalid-json" info="No snaks or invalid JSON given" /></api> --ValterVB (talk) 10:05, 22 September 2013 (UTC)[reply]

When I paste your JSON (value of &snaks=) into jsonlint.com it says:

Parse error on line 10:
...    }        }    }
---------------------^
Expecting ',', ']'

Legoktm (talk) 17:57, 24 September 2013 (UTC)[reply]

Thanks a lot. Now it work. The examples on ApiSandbox for wbsetreference are wrong. Open the square bracket but don't close it. It's possible correct it? --ValterVB (talk) 18:38, 24 September 2013 (UTC)[reply]

The examples were fixed in gerrit:86829 -- SPage (WMF) (talk) 04:13, 4 September 2015 (UTC)[reply]

ApiException in DotNetDataBot

I catched an ApiException when I try to setSiteLink.

Stack trace:

  в DotNetDataBot.Item.setSiteLink(String lang, String title, String summary)
  в DotNetDataBot.Item.setSiteLink(String lang, String title)
  в Test.Program.Main(String[] args) в E:\С-sharp\Проекты\WikiProjects\MainWikiWorker\Test\Program.cs:строка 38
  в System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
  в System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
  в Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
  в System.Threading.ThreadHelper.ThreadStart_Context(Object state)
  в System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
  в System.Threading.ThreadHelper.ThreadStart()

Message is: {"* Sorry! We could not process your edit due to a loss of session data.\nPlease try again.\nIf it still does not work, try logging out and logging back in.\n* There seems to be a problem with your login session;\nthis action has been canceled as a precaution against session hijacking.\nGo back to the previous page, reload that page and then try again.\n"}

Error code: badtoken.

I find that my token is empty when I try to post data to Wikidata. Does any reason why it's empty? --Dymitr (talk) 12:35, 5 October 2013 (UTC)[reply]

: ~~Probably~~ the problem is with http: https. When you connect to wikidata you must to use Site WD = new Site("https://www.wikidata.org", user, password); --ValterVB (talk) 14:45, 5 October 2013 (UTC)[reply]

Thanks a lot! Now it's working. --Dymitr (talk) 15:00, 5 October 2013 (UTC)[reply]

AttributeError: 'module' object has no attribute 'DataPage'

Hello,

When I try the example "Example 1: Get data" I get the error: AttributeError: 'module' object has no attribute 'DataPage' Dom (talk) 05:17, 9 October 2013 (UTC)[reply]

Using the package/pywikipedia/compat.zip and not ~~package/pywikipedia/core.zip~~ it works. Dom (talk) 04:48, 10 October 2013 (UTC)[reply]

Exact same problem here. And same solution: switching back to compat. Nicolas1981 (talk) 07:32, 11 October 2013 (UTC)[reply]

The tutorial was still for compat. I updated it now. Please test it. Instead of DataPage, the corresponding class is called ItemPage. — Felix Reimann (talk) 12:42, 11 October 2013 (UTC)[reply]

It works now, thanks! Nicolas1981 (talk) 07:33, 15 October 2013 (UTC)[reply]

How to target Commons images?

I am writing a Wikidata bot to add links to Commons images.

Problem: pywikibot.page.ImagePage(u"TheImage.jpg") does not work.

I guess I have to specify the Commons site, but how to do? Thanks! Nicolas1981 (talk) 08:05, 15 October 2013 (UTC)[reply]

I have found: site = pywikibot.Site("en", "wikivoyage"); image = pywikibot.page.ImagePage(site, u"TheImage.jpg") Nicolas1981 (talk) 08:40, 15 October 2013 (UTC)[reply]

pywikibot has no attribute 'ItemPage'

>>> import pywikibot
>>> site = pywikibot.Site("en", "wikipedia")
>>> page = pywikibot.Page(site, u"Douglas Adams")
>>> item = pywikibot.ItemPage.fromPage(page)

Traceback (most recent call last):
  File "<pyshell#23>", line 0, in <module>
    item = pywikibot.ItemPage.fromPage(page)
AttributeError: 'module' object has no attribute 'ItemPage'

Is this guide outdated or am I doing something wrong? -- Dalba 10:18, 20 November 2013 (UTC)[reply]

All Pywikibot examples are working but only with the core-branch. Maybe you are using the compat-branch of PWB. --Pasleim (talk) 10:59, 20 November 2013 (UTC)[reply]

Thanks Pasleim. You're right, I was using compat. -- Dalba 11:22, 20 November 2013 (UTC)[reply]

@Dalba, Pasleim: hi! Can you please explain me how I can change using compat-branch to core-branch?--77.121.152.181 21:59, 5 April 2020 (UTC)[reply]

datavalue for type:time

I tried to add a date with api wbeditentity, but I have message "$time needs to be a valid ISO 8601 date", so what is the correct format of a time property? Here my example that don't work. --ValterVB (talk) 11:45, 8 February 2014 (UTC)[reply]

{
   "claims": [
       {
           "mainsnak": {
               "snaktype": "value",
               "property": "P83",
               "datavalue": {
                   "value": {
                       "time": "+00000002013-10-09T00:00:00Z",
                       "timezone": 0,
                       "before": 0,
                       "after": 0,
                       "precision": 11,
                       "calendarmodel": "http://www.wikidata.org/entity/Q1985727"
                   },
                   "type": "time"
               }
           },
           "type": "statement",
           "rank": "normal"
       }
   ]
}

Solved, need url encode. --ValterVB (talk) 12:57, 8 February 2014 (UTC)[reply]

More examples for pywikibot

It would be great to have more examples : add a qualifier, create a page, how to use data.removeclaim(property, value)... Pyb (talk) 13:59, 19 March 2014 (UTC)[reply]

Redundancy

mw:Manual:Pywikibot/Wikidata already contains some useful stuff. Keeping an on-wiki documentation is fine, but... two?!? --Ricordi samoa 01:29, 24 May 2014 (UTC)[reply]

Rank

How can I edit the rank of a property through the API or Pywikibot? --Pasleim (talk) 00:46, 19 June 2014 (UTC)[reply]

You can edit with wbeditentity. --JulesWinnfield-hu (talk) 08:02, 19 June 2014 (UTC)[reply]

In pywikibot, apparently it is a parameter of the "Claim" class : class pywikibot.Claim(site, pid, rank='normal') (docs here). -CovidDatahubBot (talk) 12:54, 10 January 2021 (UTC)[reply]

A good gist on that: https://gist.github.com/urbanecm/f0b2bb95514f322207d04908cc4d8143 TiagoLubiana (talk) 12:39, 1 February 2021 (UTC)[reply]

Wikibase.NET

    Dim api As WikibaseApi = New WikibaseApi("https://www.wikidata.org", "Wikibase.NET Example/0.1")
    api.login("XXX", "YYYYY")
    Dim entityProvider As New EntityProvider(api)
    Dim languages = { "en", "cs"}
    Dim entityBySitelink As Entity = entityProvider.getEntityFromSitelink("enwiki", "Prague", languages)
    ...

getEntityFromSitelink throws exception

A first chance exception of type 'System.NotSupportedException' occurred in Wikibase.NET.dll
System.NotSupportedException: Unsupported type quantity
  v Wikibase.DataValues.DataValueFactory.newDataValue(String type, JsonValue value)
  v Wikibase.DataValues.DataValueFactory.newFromArray(JsonObject data)
  v Wikibase.Snak.newFromArray(JsonObject data)
  v Wikibase.Claim.fillData(JsonObject data)
  v Wikibase.Statement.fillData(JsonObject data)
  v Wikibase.Claim..ctor(Entity entity, JsonObject data)
  v Wikibase.Statement..ctor(Entity entity, JsonObject data)
  v Wikibase.Claim.newFromArray(Entity entity, JsonObject data)
  v Wikibase.Entity.fillData(JsonObject data)
  v Wikibase.Item.fillData(JsonObject data)
  v Wikibase.Entity..ctor(WikibaseApi api, JsonObject data)
  v Wikibase.Item..ctor(WikibaseApi api, JsonObject data)
  v Wikibase.Entity.newFromArray(WikibaseApi api, JsonObject data)
  v Wikibase.WikibaseApi.parseGetEntitiesApiResponse(JsonObject result)
  v Wikibase.WikibaseApi.getEntitesFromSitelinks(String[] sites, String[] titles, String[] languages)
  v Wikibase.EntityProvider.getEntitiesFromSitelinks(String[] sites, String[] titles, String[] languages)
  v Wikibase.EntityProvider.getEntityFromSitelink(String site, String title, String[] languages)

setSitelink issue

Hello, does anyone have an idea why item.setSitelink(page) from pywikibot example #6 results in Got an unknown error when putting data: The 'token' parameter was found in the query string, but must be in the POST body? Can it be because of using pywikipedia_compat branch? If so, how do I set a site link to a wikidata item with compat? Thanks in advance, --Thevolodymyr (talk) 20:35, 22 June 2015 (UTC)[reply]

You should change to pywikibot core. As far as I know, all examples are for core because compat is deprecated and will no longer work from July 1 on, see T101524. --Pasleim (talk) 09:34, 25 June 2015 (UTC)[reply]

"Get Q number" example didn't work -- fixed

It wasn't working because of changes in the html structure response. The code below works as of today and follows PEP8.

--PabloCastellano (talk) 00:03, 19 July 2015 (UTC)[reply]

The manual on Wikipdia recommends python pwb.py login now. Should that be changed in the manual? --Tobias1984 (talk) 11:52, 17 September 2015 (UTC)[reply]

pywikibot.page.claim object

Should the tutorial also give more detail about the claim-object? The methods are:

['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delslice__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__setslice__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']

--Tobias1984 (talk) 12:47, 17 September 2015 (UTC)[reply]

Film description in label field

Wikidata_talk:WikiProject_Movies#title: Does this come from one of the tutorials here? --- Jura 16:25, 28 December 2015 (UTC)[reply]

Add/edit property in qualifer

How add/edit property of qualifier of property source? Example item, branch: described by source (P1343) -> Great Soviet Encyclopedia (1969–1978) (Q17378135), in this need change sub-property e.g. reference URL (P854) or add new. To get list of it can use "item.claims['P1343'][1].qualifiers.items()".

On the manual is recipe "Example 8" but it only create top property, also don't work recipe from "Example 9". I trying already a few days. --Vladis13 (talk) 13:05, 2 April 2016 (UTC)[reply]

How to remove qualifier

On test page the path: claims > described by source (P1343) > Great Soviet Encyclopedia (1969–1978) (Q17378135) > section, verse, paragraph, or clause (P958). I tried like

for claim in item.claims['P1343']:
qualifier = claim.qualifiers['P958']
claim.removeQualifier(qualifier)

and other, but get error "'Claim' object has no attribute 'removeQualifier'".

Also I tried the simple case: claims > native label (P1705) > original language of film or TV show (P364), but get error "KeyError: P364" or same as above.

Manuals and wiki doesn't describes the methods removeQualifier and remove/edit Claim. --Vladis13 (talk) 16:51, 8 April 2016 (UTC)[reply]

AWB

AWB should be executed at Wikidata. Alphama (talk) 12:14, 27 June 2016 (UTC)[reply]

Login Failed error on trying to log into Bot

I have created a bot and I am trying to log into it but I am running into an issue.

My user-config.py file contents is displayed below:

from __future__ import unicode_literals

family = 'wikidata'

mylang = 'wikidata'

usernames['wikidata']['wikidata'] = u'EaasServiceBot'

I run the command on cmd prompt > python pwb.py login.py
and I get this message displayed below:

WARNING: API warning (login): Fetching a token via "action=login" is deprecated. Use "action=query&meta=tokens&type=login" instead. ERROR:Login failed (Failed)

Sharmeelaashwin (talk)
Thanks in advance.

User:EaasServiceBot doesn't exist. Matěj Suchánek (talk) 09:29, 15 January 2017 (UTC)[reply]

Decent framework to get started with?

I wish to add some data from the US Census to towns in Vermont. I cloned Wikibase.NET, but upon reading more about it, it doesn't seem to support quantities, and seems to have very limited documentation. What's a decent framework to get started with.

I have only slight exposure to Python. I've done a few other projects (not wiki related) in C# with Visual Studio.

Jc3s5h (talk) 13:01, 18 July 2017 (UTC)[reply]

You can see my framework in C# User:ValterVBot/Source, some example of use in this page and in this page. On github the source isn't update. If you have question, you can ask in my talk. --ValterVB (talk) 18:16, 18 July 2017 (UTC)[reply]

Adding comment for version history

Is it possible to add a comment for the version history with Wikibase api for PHP ? --Arch2all (talk) 17:10, 13 July 2018 (UTC)[reply]

405 Not Allowed gems/sparql-client-3.0.1

Have been using a bot to load clinical trials almost every day for past few weeks. Loaded 170,000 of 300,000 trials and bot is now prepared to link these trials to related publications, journals, investigators, organizations, MeSH & chemical codes. As of today, blocked from sparql queries. (using gem sparql-client-3.0.1. Maybe now need to register and get a key?) Not immediately obvious how to resolve. Unless there's support to identify resolution, or way to revert to previous sparql permissions), gonna need to set this aside - real-work demands being overwhelming at the moment.

Redundant `repo = site.data_repository()`

When we write site = pywikibot.Site("wikidata", "wikidata") or site = pywikibot.Site("test", "wikidata"), there is no difference between site and site.data_repository(). So repo = site.data_repository() might be redundant for most of the examples. Before modifying the examples, I wonder if there is any reason (maybe backward compatibility or something) making the examples be like this? – The preceding unsigned comment was added by Jd3main (talk • contribs) at 17:08‎, 4 October 2021 (UTC).[reply]

Indeed, it probably was the backward compatibility. --Matěj Suchánek (talk) 13:42, 12 October 2021 (UTC)[reply]

Why do we have this page and mw:Manual:Pywikibot/Wikidata

We should really only have one documentation page on how to create a pywikibot on wikidata so that it is consistent and both don't need to be maintained. This page already out-of-date as python login.py doesn't exist anymore. Lectrician1 (talk) 18:23, 21 November 2022 (UTC)[reply]