This repository, created by Webz.io, provides free datasets of publicly available news articles identified as originating from fake news websites. We release new datasets weekly, each containing approximately 1,000 articles sourced from these flagged sites. These datasets provide valuable resources for researchers, analysts, and journalists studying misinformation and disinformation trends.
Updated weekly. Sites in the datasets are verified through multiple sources. Metadata, including sentiment analysis, categories, publication dates, and source trust level is included. Covers politics, health, finance, and other key domains where misinformation is prevalent.
Each dataset is a .zip archive containing multiple JSON documents.
SciOp is part of Safeguarding Research & Culture (SRC).
Using RSS feeds full of vetted torrents, we can ensure that our cultural, intellectual and scientific heritage exists in multiple copies, in multiple places, and that no single entity or group of entities can make it all disappear.
Has a page of datasets it's protecting and a large number of RSS feeds tracking the health of other data sets (i.e., have they disappeared or not).
Public data dumps from US National Archives and US Government Publishing Office (https://www.archives.gov/). Metadata is stored in each data.json file.
The datasets themselves are stored with git-lfs so be sure you have that installed before you clone that repo.
DataHoarding.org is an index of resources and archives related to data hoarding, web archival and self hosting. It was inspired by the recent purge of online information by govenment agencies, corporations and others, and aims to provide easier access to tools and information. The goal is not only to hoard data, but parse and index it as well.
Among the mirrors they have:
- FEMA
- kernel.org
- Kiwix
- NOAA
- textfiles.com
- Youtube
MassMove is a group effort to sway public opinion towards the interests of the masses. There are multiple repositories in multiple languages.
To make progress against the pressing problems the world faces, we need to be informed by the best research and data. Our World in Data makes this knowledge accessible and understandable, to empower those working to build a better world.
Aleph is a powerful tool for people who follow the money. It helps investigators to securely access and search large amounts of data - no matter whether they are a government database or a leaked email archive.
Requires a (free?) account?
Consider adding to Searx?
This list is inspired by awesome public datasets, but for real-time datasets and sources. Normally accessed via HTTP or Websockets.
The list is separated into Free and Paid and broken into subsections based on loose categories.
HaveIBeenTrained uses clip retrieval to search the Laion-5B and Laion-400M image datasets. These are currently the largest public text-to-image datsets, and they are used to train models like Stable Diffusion, Imagen, among many others.
When it's time to train a generative AI system, organizations like Stability use those datasets to download the images from their links and present them to the model with their captions.
With HaveIBeenTrained, artists can search these databases for links to their work and flag them for removal. We partner with Laion, who built these datasets, to remove those links. This helps ensure that future models will not be trained with work that has been opted out.
Useful resources for using IPFS and building things on top of it.
A repository for monitoring attack vectors mentioned in the billion-dollar disinformation campaign to reelect the president in 2020. Includes some Python code for analyzing the data.
A topic-centric list of high-quality open datasets in public domains. By everyone, for everyone!
This website generates random JSON documents, suitable for use as test data or learning how to write and interface with various APIs.
Free datasets made available by Amazon. Stuff like an atlas of the Galactic Plane, NASA NEX data, the Human Microbiome Project, the Enron emails, Freebase, and the Marvel Universe's socialgraph. The Google Books Ngrams corpus is in here, also, alongside the Westbury Lab USENET corpus.
An opensource tool for the visualization of extremely large datasets, like twitter maps or email databases.
The datarefuge website. Probably as official as it's going to get. Has some useful definitions, at least. opendata Also has a bunch of rescued datasets available for download. data
socnet and archive of public and open data for research and study. Encourages people to upload their own datasets for others to use. I use Github to authenticate.
Vast collections of data suitable for training and teaching AI ML software.