How to Do Your Own Research: European Public Records
First in a series helping readers commit the sin of "doing their own research," featuring a Q&A with Paul Holden, a South Africa-born citizen of the UK who teaches civic investigation
Americans are spoiled on the transparency front compared to other nations. We’re discovering the joys of USASpending.gov and have long enjoyed the benefits of public access laws like FOIA. Even some of our commercial tools, like the legal search site PACER, offer features for professional and amateur investigators not found overseas. But it’s hard for many of us to know where to begin, if you want to research the affairs of U.S. allies across the pond.
We polled journalists and tech professionals about how best to find just about anything in Europe and the UK. The near-universal first response featured “Tenders Electronic Daily,” or https://ted.europa.eu/en/, an EU portal that requires a little practice and the occasional whack on the hood to get good results. Similarly, the EU’s transparency register site gets good reviews for lobbying information.
Anyone can look up search engines on the web. “Do Your Own Research” depends on a seminar approach, with starting help from a practiced pro. In this case, sometimes-contributor Paul Holden, who’s taught this subject before, walked me through a list of search tools I ended up calling “arousing.” With photos and links, Paul’s advice is below, in Q&A form. As always, if you want to add to the list, write to Library@Racket.News.
Matt Taibbi: What kinds of searches are fruitful with EU or EC bureaucracies?
Paul Holden: Basically, the best one is what you’ve already mentioned, which is the ted.europa one. The way that database works is basically, every major procurement decision that’s made by an EU company gets published in this weird EU journal. TED is like the digital version of that journal. It has a record of essentially every contract that’s been awarded. You can find a crazy amount of information. I would say it’s pretty much the only place you’d look for contracting information in the EU.
Matt Taibbi: There’s a bit of a catch, or a trick.
Paul Holden: The thing to bear in mind there is you’ve always got to do an advanced search. If you just search the top-line TED for notices it’ll only give you active notices. That’s the only thing that is currently being advertised in TED database. But if you click on Advanced Search, you can then click on All Notices.
That’s everything that’s been published by the EU since the beginning, and that’s when you get the crazy contracting information.
Then you just do a keyword search. That gives probably the most transparent stuff in terms of the amount of information they give you. So they’ll give you the awarding country, the government department in the awarding country that’s awarding the contract, who it’s been awarded to, what the criteria for the award was, what the scoring system was.
There’ll generally be contact emails and stuff like that. So if you want to follow up on a contract that’s been awarded, you can write to the contracting department and be like, “Why was this awarded or what was this for?” Or something like that.
I used TED ages ago because I was basically mapping every contract that had been awarded to an Italian arms manufacturer or Senex. And if you type in Senex, you’re getting thousands of results.
Matt Taibbi: If I look up Lockheed, am I going to get 10 billion things?
Paul Holden: You probably get a couple thousand would be my guess, at least.
Matt Taibbi: Let’s take a look here. Yeah, 187. Hm.
Paul Holden: You think there’d be more. I do find with defense companies, like when we did the Senex thing, not everything was there. As in we did a separate FOI, to the government’s departments here in the UK, and we found that there was stuff they hadn’t been publishing to the tender database for arms stuff in particular. It wasn’t super complete.
Paul Holden: I don’t know the EU so well, I know obviously the UK much better.
Matt Taibbi: What in the UK is useful at all in any way? Is there a site you’d call cool?
Paul Holden: There’s a Contracts Finder service for the UK, at https://www.gov.uk/contracts-finder. So, it’s a similar thing to what the tender database is:
Matt Taibbi: I see. And it’s a dot-uk site?
Paul Holden: The government basically moved all of its search functions for business stuff onto this gov.uk website. Everything that you’ll see, which is a government-related entity that you could search through, it’s going to be gov.uk.
There are two different things: a tender and a contract, under a certain amount. And then tender is above that. And they’ve got a couple hundred thousand notices each.
Matt Taibbi: Excellent. That’s useful. I can tell you want to tell me about other sites, not these ones I’m asking about.
Paul Holden: The one that’s really good for public information in the UK is called WhatDoTheyKnow.com.
Matt Taibbi: WhatDoTheyKnow.com. Gotcha.
Paul Holden: It’s the only standardized, centralized database for such freedom of information requests.
Matt Taibbi: For British freedom of information requests.
Paul Holden: For the whole of the UK.
Matt Taibbi: Fantastic. Can you file your requests from here, too? It looks like you can.
Paul Holden: I don’t know if there’s actually an electronic submission there, but the information is on the site.
Matt Taibbi: It looks like an all-UK version of our FOIA portals, where you can find the results of everyone else’s searches.
Paul Holden: It’s very useful. And the people who run WhatDoTheyKnow, also run a thing called mySociety:
Matt Taibbi: mySociety?
Paul Holden: Yeah. Which is just mysociety.org. They’re pretty good. They run WhatDoTheyKnow. It has information about how to reach out to elected officials. The other one that’s really good, it’s called TheyWorkForYou, which is basically a centralized database for what happens in Parliament.
The good thing there is you can search for MPs. Let just search “Keir Starmer.” Let’s see what comes up.
Matt Taibbi: Promising.
Paul Holden: The main thing there is they’ll have everything he said in Hansard, everything he said in Parliament. But also, generally speaking, they will have a register of member’s interests.
Matt Taibbi: What is that, a “Register of member’s interest?”
Paul Holden: I don’t know how it works in the States, but in the UK, if you are in Parliament, you have to register if you’re doing a second job. That’ll be in the Register of Member’s Financial Interests. But it’s not super easy to find on the parliamentary website all the time, so I do really tend to do their work. There is a public register of lobbyists in the UK.
Matt Taibbi: Is that the “Office of the Registrar of Consultant Lobbyists?”
Paul Holden: It’s a shit registry. Lobbying in the UK is not nearly as well regulated or as transparent as it is in the US.
You’ll find some pretty basic information here. But you will, at least, find a little bit, if there is a lobbying firm in the UK. If it’s registered with the Office of Registrar of Consultant Lobbyists, they will have to file a courtier report saying who their clients are. That’s about it. But it’s not going to be super easy to find out who they’ve lobbied.
The thing that I always recommend, for people doing international work, is Open Corporates.
Paul Holden: Open Corporates is pretty much the best corporate registry database there is, in the world, I think. You have to sign up for an account, which is a bit annoying, but it’s free.
But then it’ll then direct you in the data, to the underlying company’s registry where data is held. And then sometimes if you click on that, you can get way more information. So basically, directly to the company’s registry for a country.
For example, as I was looking at companies in India for our project, I came across a company involved in laundering large amounts of money through South Africa. I looked it up and got basic information in Open Corporates, but then it gave me a link to the specific registry in India where that company was registered. When I clicked through there, suddenly I found a wealth of information.
Matt Taibbi: Yeah, it’s fantastic. No, I’m looking at one now and it tells you who the controlling owners are, what the related companies are. This is great. I was looking at a pretty obscure British firm the other day called Zinc Network and this has more than I could find:
Paul Holden: Obviously, it’s limited by how much each jurisdiction discloses. Some are better than others. So if you’re going to search for Switzerland, you’re not going to find a huge amount, but you’re still going to find lots of stuff. The US ones are pretty good. UK is decent, but it’s all basic. This is the best centralized company database there is.
Matt Taibbi: Do you have anything like PACER? Is there anything like that for the EU or the EC, or for UK?
Paul Holden: It’s really shit in the UK. There isn’t a PACER in the UK. It just doesn’t exist. There is a search function for EU case law. It’s called CURIA.
Matt Taibbi: Curia is a creepy name. Pope vibes.
Paul Holden: It’s massively privatized, the court information that you can get in the EU. So basically, unless you’re using LexisNexis, you’re not going to get a huge amount.
The one thing I would recommend, that I use a lot is WorldLII.org. It’s going to take you to a super shonky-looking website, World Legal Information Institute. It looks very old-school. It’s a non-profit, and it collates information so you can learn about the legal systems around the world.
Matt Taibbi: Kind of an eyesore. Like something from the nineties.
Paul Holden: Yeah… if you look on the left-hand side, it’ll say Global, All Countries, All Databases, All Regions. Right?
Matt Taibbi: Yep.
Paul Holden: The thing that’s useful there is that certain countries, in certain places around the world, have their own version of this LII. Weirdly enough, for example, there is a SAFLII, which is the Sub-Saharan African Legal Information Institute. It has crazy amounts of stuff.
Matt Taibbi: So you have to experiment to find it.
Paul Holden: I went to find all the high court results for South Africa. In the UK, basically the only open-source searching system you can do for court judgments in the UK is their version of this, which is called BAILII.
Matt Taibbi: BAILII?
Paul Holden: It is BAILII.org. What BAILII does is that when there’s an interesting judgment at the high court or in a competition court, a judge might send it to BAILII and it’ll upload it.
And it’s been maintained by legal nerds, so you’ll have good case law in there. But it’s not like... When I came across PACER for the first time, I was like, “This is the greatest thing I’ve ever come across in my life.”
Matt Taibbi: The key thing with PACER is the party search.
Paul Holden: But there isn’t anything like that. There basically isn’t anything like PACER anywhere else in the world. PACER is pretty much globally unique.
But this is still good. You’ll find interesting stuff there. So I was looking into a corruption case in India and I had to find arrest warrants there, which is a very long story. But I went to WorldLII, I clicked on their database, I looked at India. And then I basically followed the links I had to the Delhi High Court, and then I just farted around their website until I could find something. And that’s pretty much what you have to do to find legal records. Start at WorldLII, find the country, find the court, and see what you can find.
Matt Taibbi: That’s… haphazard.
Paul Holden: It is very haphazard. The American commitment to transparency is not matched around the world, let me just tell you.
One other thing is I know that they are getting a lot of shit in the US around USA stuff at the moment, but OCCRP, they run this thing called the Catalogue of Research Databases, which is really useful to have a look at.
Paul Holden: You use the drop-down to select any country in the world. And it’s pretty high-level stuff, but it’ll give you... Like, you click on Nepal and it’ll give you the link to the register where the company is, right?
Let’s click on Ukraine. What comes up? The State Enterprise Information Resource Center, which is the official registry for Ukrainian companies. The Ukrainian Stock Exchange, unofficial database of citizens in Russia and Ukraine.
Matt Taibbi: Interesting. It takes some patience, but Burisma results spill out eventually.
Paul Holden: Basically, what they’ve done is they’ve searched pretty much every database there is in every country that vaguely faces the public, and they put that there. So, it’s a good place to start. It, at least, finds the databases themselves.
Then there’s one really good one. It’s called the OSINT Framework. It’s osintframework.com.
Paul Holden: It’s not super complete. It’s basically somebody’s project. But it’s a sprawling list of all the different online databases researchers found around different topics.
The ones that I find interesting is people search engines. And then you click on general people search engine because you see all the different databases that you can use to search people.
Matt Taibbi: Oh my god. This is a bit arousing.
Paul Holden: Right? And then public record. With all those sorts of stuff, you have to combine stuff with public records, and you click on like US voter records. What’s going to come up there? But I think it’s quite an interesting thing to use because, A, it’s good to just see what the database is that this person’s included. But also, a good way of thinking about the different categories of OSINT open data that you can search for.
Matt Taibbi: Looks like a great resource.
Paul Holden: And then there’s OCCRP Aleph.
Paul Holden: Basically, Aleph is them scraping public registries of all sorts of different shit, putting it datasets that you can search. It’s just like an information dump. It’s like a mixture of company registries, and leaks that have happened. I think WikiLeaks is on here, or some of the WikiLeaks State Department stuff was on here at some point. It’s basically, you want to find out what random shit on the internet actually looks like for you.
Matt Taibbi: Paul, I should have called you years ago.
Paul Holden: You know what else I would recommend? Have you ever looked at GIJN?
Paul Holden: That’s the Global Investigative Journalism Network. They have a resource center.
Matt Taibbi: Oh, GIJN. I’ve used this one before.
Paul Holden: The resource center is quite useful just because they have databases and a couple of how-to guides and stuff. They have resources on corruption. There is a whole set of databases you can have a look at, and it recommends all the things you want to look at if you’re researching corruption, which is quite useful.
Matt Taibbi: All right, Paul. Thank you so much. This is fantastic. Thank you for taking the time. I’m sure our readers appreciate it.
Paul Holden: No worries. Cheers!
 | A guest post by
|
What a great project this is. A service to humanity in this early infantile digital age.
Shamed for Doing Your Own Research: Jimmy Dore Brilliantly Breaks Down the COVID Propaganda: https://old.bitchute.com/video/4DdN99KD76JO [1:45mins]
From that hilarious video:
Someone says: "I'm going to go buy a car"
COVID logic replies: "Don't look into it. Ask the salesman he's the expert. What are you, Henry Ford?"