unique_name stringlengths 3 45 | dataset_year stringdate 1980-01-01 00:00:00 2025-01-01 00:00:00 | domain_str stringclasses 11
values | dataset_source stringclasses 5
values | original_dataset_source_download_link stringlengths 29 91 | download_description stringlengths 171 1.06k | academic_reference_bibtex stringlengths 112 732 | academic_reference_bibtex_key stringlengths 11 49 | license stringclasses 9
values | data_tags listlengths 1 3 | curation_comments stringlengths 4 2.41k | version_from_unique_name null | version_comment null | local_data_directory_base stringclasses 2
values | type_adapter_id stringclasses 1
value |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
allstate_claims_severity | 2016 | insurance | Kaggle | https://www.kaggle.com/competitions/allstate-claims-severity |
We use the train.csv from the Kaggle competition.
kaggle competitions download -c allstate-claims-severity -f train.csv.zip && unzip train.csv.zip && rm train.csv.zip
mkdir -p local-data-warehouse/allstate_claims_severity && mv train.csv local-data-warehouse/allstate_claims_severity/
| @misc{Ferguson2016AllstateClaimsSeverity,
author = {Dana Ferguson and Meg Risdal and NoTrick and Sara R. Sillah and Tim Emmerling and Will Cukierski},
title = {Allstate Claims Severity},
year = {2016},
howpublished = {\url{https://kaggle.com/competitions/allstate-claims-severity},
note = {Kaggle competit... | Ferguson2016AllstateClaimsSeverity | Kaggle Competition Rules | [
"IID",
"Anonymized"
] |
We start with the train.csv from Kaggle.
- The data has been anonymized.
- Top Kaggle solutions did not perform any relevant preprocessing.
- We drop the ID column, as it does not contain any signal.
- The data contains 1 duplicate row when ignoring the target. We drop this artifact.
- We log scale the target.
| null | null | /work/dlclarge1/matusd-data_foundry/data-foundry/local-data-warehouse | dataset-mold-v1 |
ancestry_study | 1990 | biology & life sciences | Other | https://github.com/CarolaHeinzel/BGA_Classification/blob/main/data/input_data/full_data.csv |
We download the data from the GitHub repository and save it to a predefined folder.
mkdir -p local-data-warehouse/ancestry_study/ && curl -L -o local-data-warehouse/ancestry_study/filter_population.xlsx https://raw.githubusercontent.com/CarolaHeinzel/BGA-Classification/main/datat/filtered_population_eur_update.xlsx
| @article{ruiz2023ancestry,
title={Development and evaluations of the ancestry informative markers of the VISAGE Enhanced Tool for Appearance and Ancestry},
author={Ruiz-Ram{'\i}rez, Jorge and de La Puente, M and Xavier, Catarina and Ambroa-Conde, Adri{'a}n and {'A}lvarez-Dios, J and Freire-Aradas, A and Mosquera-Mi... | ruiz2023ancestry | Public | [
"IID"
] |
-
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
anneal | 1990 | industry & manufacturing | UCI | https://archive.ics.uci.edu/dataset/3/annealing |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/anneal/ && wget -P local-data-warehouse/anneal/ https://archive.ics.uci.edu/static/public/3/annealing.zip && unzip local-data-warehouse/anneal/annealing.zip -d local-data-warehouse/anneal/
| @misc{uci1990annealing,
title = {Annealing},
year = {1990},
author = {Unknown},
howpublished = {UCI Machine Learning Repository},
url = {https://doi.org/10.24432/C5RW2F},
}
| uci1990annealing | CC BY 4.0 | [
"IID"
] |
- We combined train and test data into a single dataset.
- We replaced '?' with 'not_applicable' as described in the metadata.
- We renamed some features to remove '/' characters.
- Anomaly: In the original data, class 4 is in the metadata, but there are no samples in this class.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
aps_failure | 2016 | industry & manufacturing | UCI | https://doi.org/10.24432/C5V60Q |
mkdir -p local-data-warehouse/aps_failure
wget -P local-data-warehouse/aps_failure/ https://archive.ics.uci.edu/static/public/414/ida2016challenge.zip
unzip local-data-warehouse/aps_failure/ida2016challenge.zip -d local-data-warehouse/aps_failure/
rm local-data-warehouse/aps_failure/ida2016challenge... | @misc{ida2016challenge,
author = {{IDA2016Challenge}},
title = {IDA2016Challenge [Dataset]},
year = {2016},
howpublished = {\url{https://doi.org/10.24432/C5V60Q}},
note = {UCI Machine Learning Repository},
}
| ida2016challenge | CC BY 4.0 | [
"ForcedIIDFromTemporal",
"Anonymized"
] |
- We combined the original training and testing data into a single dataset.
- We renamed the target feature to "AirPressureSystemFailure".
- We converted "na" strings to real NaN/missing values, making the data numeric.
- Anomaly: we cannot determine the data types of the features.
- Anomaly: some features a... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
audiology_diagnosis | 1987 | medical & healthcare | UCI | https://doi.org/10.24432/C5TP4R |
We get the data from UCI.
wget https://archive.ics.uci.edu/static/public/8/audiology+standardized.zip && unzip audiology+standardized.zip && rm audiology+standardized.zip audiology.standardized.names && mkdir -p local-data-warehouse/audiology_diagnosis && mv audiology.standardized.data audiology.standardized.test loca... | @incollection{bareiss1990protos,
title={Protos: An exemplar-based learning apprentice},
author={Bareiss, E Ray and Porter, Bruce W and Wier, Craig C},
booktitle={Machine learning},
pages={112--127},
year={1990},
publisher={Elsevier}
}
| bareiss1990protos | CC BY 4.0 | [
"IID"
] |
We start with the data from UCI and merge train and test data.
- We drop the ID column as it is uninformative here.
- The target label contains groups of labels with specifications. We merge labels into groups that represent general a diagnosis. We create 3 labels: "normal", "cochlear", "other". There is likely a muc... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
automobile | 1985 | industry & manufacturing | UCI | https://doi.org/10.24432/C5B01C |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/automobile/ && wget -P local-data-warehouse/automobile/ https://archive.ics.uci.edu/static/public/10/automobile.zip && unzip local-data-warehouse/automobile/automobile.zip -d local-data-warehouse/automobile/... | @misc{Schlimmer1985Automobile,
title={Automobile},
author={Jeffrey Schlimmer},
year={1985},
title={1985 Model Import Car and Truck Specifications},
journal={Ward's Automotive Yearbook}
}
| Schlimmer1985Automobile | CC BY 4.0 | [
"IID"
] |
- We rename the target feature from "symboling" to "risk_factor" for clarity of the task.
- We remove "normalized-losses" attribute as it has been handcrafted and might lead to data leaks.
- We encode missing values as np.nan instead of "?".
| null | null | /work/dlclarge1/matusd-data_foundry/data-foundry/local-data-warehouse | dataset-mold-v1 |
bad_customer_detection | 2020 | business & marketing | Kaggle | https://www.kaggle.com/datasets/podsyp/is-this-a-good-customer |
Download the dataset from Kaggle using the Kaggle API.
mkdir -p local-data-warehouse/bad_customer_detection/ && cd local-data-warehouse/bad_customer_detection && kaggle datasets download podsyp/is-this-a-good-customer && unzip is-this-a-good-customer.zip && rm is-this-a-good-customer.zip && cd ../../
| @misc{Podsyp2020IsThisAGoodCustomer,
title={Is This a Good Customer?},
author={Podsyp},
year={2020},
publisher={Kaggle},
url={https://www.kaggle.com/datasets/podsyp/is-this-a-good-customer}
}
| Podsyp2020IsThisAGoodCustomer | Public Domain | [
"IID"
] |
- We shuffle the data as it has an original distribution shift.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
bank_customer_churn | 2020 | business & marketing | Kaggle | https://www.kaggle.com/datasets/gauravtopre/bank-customer-churn-dataset |
We download the data from Kaggle to a .csv file in a predefined folder.
mkdir -p local-data-warehouse/bank_customer_churn && cd local-data-warehouse/bank_customer_churn && kaggle datasets download gauravtopre/bank-customer-churn-dataset && unzip bank-customer-churn-dataset.zip && rm bank-customer-churn-dataset.zi... | @misc{Topre2022BankCustomerChurn,
title={Bank Customer Churn Dataset},
author={Gaurav Topre},
year={2022},
publisher={Kaggle},
url={https://www.kaggle.com/datasets/gauravtopre/bank-customer-churn-dataset}
}
| Topre2022BankCustomerChurn | Public Domain | [
"IID"
] |
- We remove the customer_id column.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
bank_marketing | 2012 | finance | UCI | https://archive.ics.uci.edu/dataset/222/bank+marketing |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/bank_marketing/ && wget -P local-data-warehouse/bank_marketing/ https://archive.ics.uci.edu/static/public/222/bank+marketing.zip && unzip -o "local-data-warehouse/bank_marketing/bank+marketing.zip" -d local-... | @article{moro2014bank-marketing,
title={A data-driven approach to predict the success of bank telemarketing},
author={Moro, S{'e}rgio and Cortez, Paulo and Rita, Paulo},
journal={Decision Support Systems},
volume={62},
pages={22--31},
year={2014},
publisher={Elsevier}
}
| moro2014bank-marketing | CC BY 4.0 | [
"IID"
] |
- We removed the "duration" feature following its original description to obtain a "realistic predictive model".
- We further remove the "month" and "day_of_week" features, as they also relate to the last contact -- which is not available in a real-world scenario.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
bioresponse | 2012 | biology & life sciences | Kaggle | https://www.kaggle.com/c/bioresponse |
We download the data from Kaggle and uzip it to a predefined folder.
mkdir -p local-data-warehouse/bioresponse && cd local-data-warehouse/bioresponse && kaggle competitions download -c bioresponse && unzip bioresponse.zip && rm bioresponse.zip
| @misc{bioresponse2012hamner,
author = {Ben Hamner and dcthompson and Jorg},
title = {Predicting a Biological Response},
year = {2012},
howpublished = {https://kaggle.com/competitions/bioresponse},
note = {Kaggle}
}
| bioresponse2012hamner | Public Domain | [
"IID"
] |
- We changed the name of the target and mapped it to yes/no
- Anomaly: Only train data is used, since test data is empty for target
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
body_density_prediction | 1985 | medical & healthcare | Kaggle | https://www.kaggle.com/datasets/fedesoriano/body-fat-prediction-dataset |
We download the data from Kaggle.
kaggle datasets download fedesoriano/body-fat-prediction-dataset && unzip body-fat-prediction-dataset.zip && rm body-fat-prediction-dataset.zip
mkdir -p local-data-warehouse/body_density_prediction && mv bodyfat.csv local-data-warehouse/body_density_prediction/
| @article{penrose1985generalized,
title={Generalized body composition prediction equation for men using simple measurement techniques},
author={Penrose, Keith W and Nelson, Arnold G and Fisher, Arnold Garth},
journal={Medicine \& Science in Sports \& Exercise},
volume={17},
number={2},
pages={189},
year={1... | penrose1985generalized | None | [
"IID"
] |
We start with the data from Kaggle.
- Note, the task is not about bodyfat prediction as this is determined by a deterministic formula. Instead, the task is to estimate the density (which can be used to get the body fat via the deterministic formula). However, density requires a test. So we aim to predict from data th... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
brasilian_houses | 2020 | finance | OpenML | https://www.openml.org/data/download/22044541/dataset |
We download the data from OpenML.
In this notebook, run:
import openml
# Load the dataset object from OpenML
dataset = openml.datasets.get_dataset(
42688,
download_data=True,
download_qualities=False,
download_features_meta_data=False,
)
dataset.get_data()[0].... | @misc{farias2020brasilian,
author = {Fernando Lucas de Oliveira Farias and Raphael Fontes},
title = {Ensino 5A > Datacamp},
year = {2020},
howpublished = {\url{https://kaggle.com/competitions/ensino-5a-datacamp}},
note = {Kaggle}
}
| farias2020brasilian | Public domain | [
"IID"
] |
- We start with the data from Kaggle and try to get the data from the competition, but it is a limited-participation competition. Only invited users may participate. So instead, we searched for other sources and found https://www.kaggle.com/datasets/antonyroy/finaltry/data and https://www.openml.org/d/42688, which se... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
churn | 2005 | technology & internet | OpenML | https://www.openml.org/d/40701 |
We download the data from OpenML.
In this notebook, run:
import openml
# Load the dataset object from OpenML
dataset = openml.datasets.get_dataset(
40701,
download_data=True,
download_qualities=False,
download_features_meta_data=False,
)
dataset.get_data()[0].... | @misc{marcoulides2005churn,
title={Discovering knowledge in data: An introduction to data mining},
author={Marcoulides, George A},
year={2005},
publisher={Taylor \& Francis}
}
| marcoulides2005churn | Public Domain | [
"IID"
] |
- The original source is lost, so we use https://www.openml.org/d/40701 (or https://github.com/EpistasisLab/pmlb/tree/master/datasets/churn)
- We dropped the "phone_number" feature as it seems to be an index in the original data.
- We renamed the target variable to "CustomerChurned" and mapped binary variables to "Ye... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
clock_protein_period | 2021 | biology & life sciences | UCI | https://doi.org/10.24432/C5B31D |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/clock_protein_period && wget -P local-data-warehouse/clock_protein_period/ https://archive.ics.uci.edu/static/public/729/period+changer-2.zip && unzip local-data-warehouse/clock_protein_period/period+change... | @article{Gul2021StructureBasedDC,
title={Structure-based design and classifications of small molecules regulating the circadian rhythm period},
author={Sadia Gul and Fatima Rahim and Selen Isin and others},
journal={Scientific Reports},
year={2021},
volume={11},
pages={18510},
doi={10.1038/s41598-021-9796... | Gul2021StructureBasedDC | CC BY 4.0 | [
"IID"
] |
- We remove duplicated columns but keep the first occurrence.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
clock_protein_toxicity | 2021 | biology & life sciences | UCI | https://doi.org/10.24432/C59313 |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/clock_protein_toxicity && wget -P local-data-warehouse/clock_protein_toxicity/ https://archive.ics.uci.edu/static/public/728/toxicity-2.zip && unzip local-data-warehouse/clock_protein_toxicity/toxicity-2.zi... | @article{gul2021structure,
title={Structure-based design and classifications of small molecules regulating the circadian rhythm period},
author={Gul, Seref and Rahim, Fatih and Isin, Safak and Yilmaz, Fatma and Ozturk, Nuri and Turkay, Metin and Kavakli, Ibrahim Halil},
journal={Scientific reports},
volume={11}... | gul2021structure | CC BY 4.0 | [
"IID"
] |
- We remove duplicated columns (same values for all rows).
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
coil_2000_insurance_policies | 2000 | business & marketing | UCI | https://doi.org/10.24432/C5630S |
We download the data from the UCI repository and unzip it to a predefined folder.
mkdir -p local-data-warehouse/coil_2000_insurance_policies/ && wget -P local-data-warehouse/coil_2000_insurance_policies/ https://archive.ics.uci.edu/static/public/125/insurance+company+benchmark+coil+2000.zip && unzip local-data-wareho... | @techreport{van2000coil,
title={CoIL challenge 2000: The insurance company case},
author={Van Der Putten, Peter and van Someren, Maarten and others},
year={2000},
institution={Technical Report 2000--09, Leiden Institute of Advanced Computer Science}
}
| van2000coil | CC BY 4.0 | [
"IID"
] |
- We created semantic meaningful names for the features.
- We combined the original training and validation data into one new dataset.
- We reversed the ordinal encoding of the original data where possible.
- Anomaly: the data has 15% duplicates.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
colon_tumor | 1999 | medical & healthcare | OpenML | https://www.openml.org/d/45087 |
We download the data to a predefined folder.
mkdir -p local-data-warehouse/colon_tumor && wget -P local-data-warehouse/colon_tumor https://api.openml.org/data/download/22112148/dataset
| @article{Alon1999BroadPO,
title={Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays},
author={Uri Alon and Naama Barkai and Daniel A. Notterman and Kurt Gish and Suzanne Ybarra and Daniel Mack and Arnold J. Levine},
journal={Proceed... | Alon1999BroadPO | Public Domain | [
"IID"
] |
- We use the OpenML version of the data as original raw data in not available. The features have been pre-filtered in an unsupervised way from 6,500 to 2,000. They have been already z-normalized.
- We remove 9 duplicated columns: "att_261", "att_262", "att_263", "att_51", "att_52", "att_53", "att_40", ... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
credit_approval | 1987 | finance | UCI | https://doi.org/10.24432/C5FS30 |
We get the credit screening data from UCI.
wget https://archive.ics.uci.edu/static/public/27/credit+approval.zip && unzip credit+approval.zip crx.data && rm credit+approval.zip && mkdir -p local-data-warehouse/credit_approval && mv crx.data local-data-warehouse/credit_approval/
| @article{quinlan1987simplifying,
title={Simplifying decision trees},
author={Quinlan, J. Ross},
journal={International journal of man-machine studies},
volume={27},
number={3},
pages={221--234},
year={1987},
publisher={Elsevier}
}
| quinlan1987simplifying | CC BY 4.0 | [
"IID"
] |
We get the data from UCI.
- We encode A14 as numeric, following the description of it being a continuous feature.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
credit_card_clients_default | 2009 | finance | UCI | https://doi.org/10.24432/C55S3H |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/credit_card_clients_default/ && wget -P local-data-warehouse/credit_card_clients_default/ https://archive.ics.uci.edu/static/public/350/default+of+credit+card+clients.zip && unzip local-data-warehouse/credit... | @article{yeh2009comparisons,
title={The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients},
author={Yeh, I-Cheng and Lien, Che-hui},
journal={Expert systems with applications},
volume={36},
number={2},
pages={2473--2480},
year={2009},
publ... | yeh2009comparisons | CC BY 4.0 | [
"IID"
] |
- We rename the target variable and restore the original class names.
- We drop the "ID" column.
- Anomaly: the data has temporal features but the task is time-invariant.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
customer_satisfaction_in_airline | 2023 | biology & life sciences | Kaggle | https://www.kaggle.com/datasets/yakhyojon/customer-satisfaction-in-airline |
We download the data from Kaggle and uzip it to a predefined folder.
mkdir -p local-data-warehouse/customer_satisfaction_in_airline/ && cd local-data-warehouse/customer_satisfaction_in_airline/ && kaggle datasets download yakhyojon/customer-satisfaction-in-airline && cd ../../ && unzip local-data-warehouse/customer_s... | @misc{yakhyojon2023airlinesatisfaction,
author = {Kaggle User Yakhyojon},
title = {Customer Satisfaction in Airline.},
year = {2023},
howpublished = {url{https://www.kaggle.com/datasets/yakhyojon/customer-satisfaction-in-airline}},
note = {Kaggle}
}
| yakhyojon2023airlinesatisfaction | Public Domain | [
"IID"
] |
- We renamed the target column from "satisfaction" to "satisfied" for clarity and mapped the values to "Yes" and "No"
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
drug_induced_autoimmunity_prediction | 2025 | medical & healthcare | UCI | https://doi.org/10.24432/C5332M |
wget https://archive.ics.uci.edu/static/public/1104/drug_induced_autoimmunity_prediction.zip && unzip drug_induced_autoimmunity_prediction.zip && rm drug_induced_autoimmunity_prediction.zip RDKit_ChemDes.xlsx && mkdir -p local-data-warehouse/drug_induced_autoimmunity_prediction && mv DIA_trainingset_RDKit_descriptors.... | @article{huang2025interdia,
title={InterDIA: Interpretable prediction of drug-induced autoimmunity through ensemble machine learning approaches},
author={Huang, Lina and Liu, Peineng and Huang, Xiaojie},
journal={Toxicology},
volume={511},
pages={154064},
year={2025},
publisher={Elsevier}
}
| huang2025interdia | CC BY 4.0 | [
"IID"
] |
We start with the dataset from UCI.
- We drop the constant columns.
- We keep the SMILES code as string for later pipelines to handle.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
forest_fires | 2008 | environmental science & climate | UCI | https://doi.org/10.24432/C5D88D |
We get the data from UCI.
wget https://archive.ics.uci.edu/static/public/162/forest+fires.zip && unzip forest+fires.zip && rm forest+fires.zip forestfires.names
mkdir -p local-data-warehouse/forest_fires && mv forestfires.csv local-data-warehouse/forest_fires/
| @article{cortez2007data,
title={A data mining approach to predict forest fires using meteorological data},
author={Cortez, Paulo and Morais, An{\'\i}bal de Jesus Raimundo},
year={2007},
publisher={Associa{\c{c}}{\~a}o Portuguesa para a Intelig{\^e}ncia Artificial (APPIA)}
}
| cortez2007data | CC BY 4.0 | [
"IID",
"Spatial",
"ForcedIIDFromTemporal"
] |
We start with the data from UCI.
- The original paper used a IID split. The tasks sounds like it requires a temporal split. However, the data does not contain the original time indices (or order of samples) and just month/day over 3 years. Furthermore, the authors made sure to only use time-invariant features as it s... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
framingham_heart_study | 2022 | physics & astronomy | Kaggle | https://www.kaggle.com/datasets/aasheesh200/framingham-heart-study-dataset |
We download the data from Kaggle.
mkdir -p local-data-warehouse/framingham_heart_study/ && cd local-data-warehouse/framingham_heart_study && kaggle datasets download aasheesh200/framingham-heart-study-dataset && cd ../../ && unzip local-data-warehouse/framingham_heart_study/framingham-heart-study-dataset.zip -d local... | @misc{bhardwaj2022framingham,
title={Framingham heart study dataset },
url={https://www.kaggle.com/dsv/3493583},
DOI={10.34740/KAGGLE/DSV/3493583},
publisher={Kaggle},
author={Ashish Bhardwaj},
year={2022}
}
| bhardwaj2022framingham | Public Domain | [
"IID"
] |
- We mapped binary features to yes/no
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
give_me_some_credit | 2011 | finance | Kaggle | https://www.kaggle.com/competitions/GiveMeSomeCredit/overview |
We download the data from Kaggle and uzip it to a predefined folder.
mkdir -p local-data-warehouse/give_me_some_credit/ && cd local-data-warehouse/give_me_some_credit && kaggle competitions download -c GiveMeSomeCredit && cd ../../ && unzip local-data-warehouse/give_me_some_credit/GiveMeSomeCredit.zip -d local-data-w... | @misc{cukierski2011credit,
author = {Credit Fusion and Will Cukierski},
title = {Give Me Some Credit},
year = {2011},
howpublished = {url{https://kaggle.com/competitions/GiveMeSomeCredit}},
note = {Kaggle}
}
| cukierski2011credit | Public | [
"IID"
] |
- We renamed the target feature and its value to be more descriptive.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
hazelnut_spread_contaminant_detection_10GHz | 2020 | biology & life sciences | OpenML | https://www.openml.org/d/45538 |
We download the data from OpenML.
In this notebook, run:
import openml
# Load the dataset object from OpenML
dataset = openml.datasets.get_dataset(
45538,
download_data=True,
download_qualities=False,
download_features_meta_data=False,
)
dataset.get_data()[0].... | @INPROCEEDINGS{urbinati2020hazelnut,
author={Urbinati, Luca and Ricci, Marco and Turvani, Giovanna and Vasquez, Jorge A. Tobon and Vipiana, Francesca and Casu, Mario R.},
booktitle={2020 IEEE International Symposium on Circuits and Systems (ISCAS)},
title={A Machine-Learning Based Microwave Sensing Approach to F... | urbinati2020hazelnut | CC BY-SA | [
"IID"
] |
- We select a features from a single frequency (10 GHz) as the authors also only considered this frequency for the final experiments.
- Anomaly: we use the publicly available dataset state, which is without preprocessing. Moreover, we were not able to inverse the original ordinal encoding of the label.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
hazelnut_spread_contaminant_detection_10ghz | 2020 | biology & life sciences | OpenML | https://www.openml.org/d/45538 |
We download the data from OpenML.
In this notebook, run:
import openml
# Load the dataset object from OpenML
dataset = openml.datasets.get_dataset(
45538,
download_data=True,
download_qualities=False,
download_features_meta_data=False,
)
dataset.get_data()[0].... | @INPROCEEDINGS{urbinati2020hazelnut,
author={Urbinati, Luca and Ricci, Marco and Turvani, Giovanna and Vasquez, Jorge A. Tobon and Vipiana, Francesca and Casu, Mario R.},
booktitle={2020 IEEE International Symposium on Circuits and Systems (ISCAS)},
title={A Machine-Learning Based Microwave Sensing Approach to F... | urbinati2020hazelnut | CC BY-SA | [
"IID"
] |
- We select a features from a single frequency (10 GHz) as the authors also only considered this frequency for the final experiments.
- Anomaly: we use the publicly available dataset state, which is without preprocessing. Moreover, we were not able to inverse the original ordinal encoding of the label.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
heart_disease_cleveland | 1989 | medical & healthcare | UCI | https://doi.org/10.24432/C52P4X |
Get the UCI data.
wget https://archive.ics.uci.edu/static/public/45/heart+disease.zip && unzip heart+disease.zip processed.cleveland.data && rm heart+disease.zip && mkdir -p local-data-warehouse/heart_disease_cleveland && mv processed.cleveland.data local-data-warehouse/heart_disease_cleveland/
| @article{detrano1989international,
title={International application of a new probability algorithm for the diagnosis of coronary artery disease},
author={Detrano, Robert and Janosi, Andras and Steinbrunn, Walter and Pfisterer, Matthias and Schmid, Johann-Jakob and Sandhu, Sarbjit and Guppy, Kern H and Lee, Stella a... | detrano1989international | CC BY 4.0 | [
"IID"
] |
We start with the processed version and the subset of 14 attributes used in the study and clinical practice.
- We encode missing values as np.nan instead of "?".
- We make the target binary (0=no heart disease, 1=heart disease). This follows the original study in attempting to distinguish presence (values 1,2,3,4) fr... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
heart_disease_hungary | 1989 | medical & healthcare | UCI | https://doi.org/10.24432/C52P4X |
Get the UCI data.
wget https://archive.ics.uci.edu/static/public/45/heart+disease.zip && unzip heart+disease.zip processed.hungarian.data && rm heart+disease.zip && mkdir -p local-data-warehouse/heart_disease_hungary && mv processed.hungarian.data local-data-warehouse/heart_disease_hungary/
| @article{detrano1989international,
title={International application of a new probability algorithm for the diagnosis of coronary artery disease},
author={Detrano, Robert and Janosi, Andras and Steinbrunn, Walter and Pfisterer, Matthias and Schmid, Johann-Jakob and Sandhu, Sarbjit and Guppy, Kern H and Lee, Stella a... | detrano1989international | CC BY 4.0 | [
"IID"
] |
We start with the processed version and the subset of 14 attributes used in the study and clinical practice.
- We encode missing values as np.nan instead of "?".
- We make the target binary (0=no heart disease, 1=heart disease). This follows the original study in attempting to distinguish presence (values 1,2,3,4) fr... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
heart_disease_switzerland | 1989 | medical & healthcare | UCI | https://doi.org/10.24432/C52P4X |
Get the UCI data.
wget https://archive.ics.uci.edu/static/public/45/heart+disease.zip && unzip heart+disease.zip processed.switzerland.data && rm heart+disease.zip && mkdir -p local-data-warehouse/heart_disease_switzerland && mv processed.switzerland.data local-data-warehouse/heart_disease_switzerland/
| @article{detrano1989international,
title={International application of a new probability algorithm for the diagnosis of coronary artery disease},
author={Detrano, Robert and Janosi, Andras and Steinbrunn, Walter and Pfisterer, Matthias and Schmid, Johann-Jakob and Sandhu, Sarbjit and Guppy, Kern H and Lee, Stella a... | detrano1989international | CC BY 4.0 | [
"IID"
] |
We start with the processed version and the subset of 14 attributes used in the study and clinical practice.
- We encode missing values as np.nan instead of "?".
- We make the target binary (0=no heart disease, 1=heart disease). This follows the original study in attempting to distinguish presence (values 1,2,3,4) fr... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
heart_disease_va_long_beach | 1989 | medical & healthcare | UCI | https://doi.org/10.24432/C52P4X |
Get the UCI data.
wget https://archive.ics.uci.edu/static/public/45/heart+disease.zip && unzip heart+disease.zip processed.va.data && rm heart+disease.zip && mkdir -p local-data-warehouse/heart_disease_va_long_beach && mv processed.va.data local-data-warehouse/heart_disease_va_long_beach/
| @article{detrano1989international,
title={International application of a new probability algorithm for the diagnosis of coronary artery disease},
author={Detrano, Robert and Janosi, Andras and Steinbrunn, Walter and Pfisterer, Matthias and Schmid, Johann-Jakob and Sandhu, Sarbjit and Guppy, Kern H and Lee, Stella a... | detrano1989international | CC BY 4.0 | [
"IID"
] |
We start with the processed version and the subset of 14 attributes used in the study and clinical practice.
- We encode missing values as np.nan instead of "?".
- We make the target binary (0=no heart disease, 1=heart disease). This follows the original study in attempting to distinguish presence (values 1,2,3,4) fr... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
heloc | 2021 | finance | Kaggle | https://www.kaggle.com/datasets/averkiyoliabev/home-equity-line-of-creditheloc |
We download the data from Kaggle and uzip it to a predefined folder.
mkdir -p local-data-warehouse/heloc/ && cd local-data-warehouse/heloc && kaggle datasets download averkiyoliabev/home-equity-line-of-creditheloc && cd ../../ && unzip local-data-warehouse/heloc/home-equity-line-of-creditheloc.zip -d local-data-wareh... | @misc{averkiyoliabev2021heloc,
author = {Kaggle User Averkiyoliabev},
title = {Home Equity Line of Credit (HELOC)},
year = {2021},
howpublished = {\url{https://www.kaggle.com/datasets/averkiyoliabev/home-equity-line-of-creditheloc}},
note = {Kaggle dataset},
}
| averkiyoliabev2021heloc | Public | [
"IID"
] |
- Anomaly: the dataset has time-related features. However, the task and features are preprocessed to be time-invariant.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
hepatitis_survival_prediction | 1981 | medical & healthcare | UCI | https://doi.org/10.24432/C5Q59J |
Get the UCI data.
wget https://archive.ics.uci.edu/static/public/46/hepatitis.zip && unzip hepatitis.zip hepatitis.data && rm hepatitis.zip && mkdir -p local-data-warehouse/hepatitis_survival_prediction && mv hepatitis.data local-data-warehouse/hepatitis_survival_prediction/
| @inproceedings{efron1981statistical,
title={Statistical theory and the computer},
author={Efron, Bradley and Gong, Gail},
booktitle={Computer science and statistics: Proceedings of the 13th Symposium on the Interface},
pages={3--7},
year={1981},
organization={Springer}
}
| efron1981statistical | CC BY 4.0 | [
"IID"
] |
We start with the data from UCI.
- We encode missing values as NaN.
- We convert numeric features to float and categorical features to category dtype.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
hiva_agnostic | 2007 | chemistry & material science | Other | https://www.agnostic.inf.ethz.ch |
We download the dataset from the challenge platform into predefined folder:
mkdir -p local-data-warehouse/hiva_agnostic/ && wget -P local-data-warehouse/hiva_agnostic/ https://www.agnostic.inf.ethz.ch/datasets/DataAgnos/HIVA.zip && wget -P local-data-warehouse/hiva_agnostic/ https://www.agnostic.inf.ethz.ch/datas... | @inproceedings{guyon2007agnostic,
author={Isabelle Guyon and Amir Saffari and Gideon Dror and Gavin Cawley},
booktitle={2007 International Joint Conference on Neural Networks},
title={Agnostic Learning vs. Prior Knowledge Challenge},
year={2007},
pages={829-834},
doi={10.1109/IJCNN.2007.4371065}
}
| guyon2007agnostic | Public Domain | [
"IID"
] |
- We take the original data with the original labels with three classes.
- We only take the training data from the challenge, as the original labels are not available for the validation data.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
homesite_quote_conversion | 2015 | insurance | Kaggle | https://www.kaggle.com/competitions/homesite-quote-conversion |
We use the train.csv from the Kaggle competition.
kaggle competitions download -c homesite-quote-conversion -f train.csv.zip && unzip train.csv.zip && rm train.csv.zip
mkdir -p local-data-warehouse/homesite_quote_conversion && mv train.csv local-data-warehouse/homesite_quote_conversion/
| @misc{Darrel2015HomesiteQuoteConversion,
author = {Darrel and Stephen D. Stayton and Will Cukierski},
title = {Homesite Quote Conversion},
year = {2015},
howpublished = {\url{https://kaggle.com/competitions/homesite-quote-conversion}},
note = {Kaggle competition}
}
| Darrel2015HomesiteQuoteConversion | Kaggle Competition Rules | [
"IID",
"Anonymized"
] |
We start with the train.csv from Kaggle.
- The data has been anonymized, so feature meanings are unknown.
- The data contains decoded spatial information.
- We fix the Field10 column to be integer instead of string with ',' characters.
- We convert the Original_Quote_Date to datetime.
- IMPORTANT NOTE: this dataset w... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
horse_colic_survival | 1989 | biology & life sciences | UCI | https://doi.org/10.24432/C58W23 |
Get the train and test data from UCI.
wget https://archive.ics.uci.edu/static/public/47/horse+colic.zip && unzip horse+colic.zip horse-colic.data horse-colic.test && rm horse+colic.zip && mkdir -p local-data-warehouse/horse_colic_survival && mv horse-colic.data horse-colic.test local-data-warehouse/horse_colic_surviv... | @misc{McLeish1989HorseColic,
author = {McLeish, Mary and Cecile, Matt},
title = {{Horse Colic}},
year = {1989},
howpublished = {UCI Machine Learning Repository},
note = {{DOI}: https://doi.org/10.24432/C58W23}
}
| McLeish1989HorseColic | CC BY 4.0 | [
"IID"
] |
We start with the data from UCI and merge the train and test data.
- We encode missing values as NaN.
- We drop features that leak the target.
- We keep only the first instance (hospital number) per horse, as they can be treated multiple times and otherwise leaks information about the target.
- We use the target "out... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
hr_analytics | 2021 | business & marketing | Kaggle | https://www.kaggle.com/datasets/arashnic/hr-analytics-job-change-of-data-scientists |
mkdir -p local-data-warehouse/hr_analytics && cd local-data-warehouse/hr_analytics && kaggle datasets download arashnic/hr-analytics-job-change-of-data-scientists && unzip hr-analytics-job-change-of-data-scientists.zip && rm hr-analytics-job-change-of-data-scientists.zip && cd ../../
| @misc{Arashnic2021HRAnalytics,
title={HR Analytics: Job Change of Data Scientists},
author={Arashnic},
year={2021},
publisher={Kaggle},
url={https://www.kaggle.com/datasets/arashnic/hr-analytics-job-change-of-data-scientists}
}
| Arashnic2021HRAnalytics | Public Domain | [
"IID"
] |
- We remove the "employee_id" column.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
in_vehicle_coupon_recommendation | 2017 | business & marketing | UCI | https://doi.org/10.24432/C5GS4P |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/in_vehicle_coupon_recommendation/ && wget -P local-data-warehouse/in_vehicle_coupon_recommendation/ https://archive.ics.uci.edu/static/public/603/in+vehicle+coupon+recommendation.zip && unzip local-data-ware... | @article{wang2017bayesian,
title={A bayesian framework for learning rule sets for interpretable classification},
author={Wang, Tong and Rudin, Cynthia and Doshi-Velez, Finale and Liu, Yimin and Klampfl, Erica and MacNeille, Perry},
journal={Journal of Machine Learning Research},
volume={18},
number={70},
pa... | wang2017bayesian | CC BY 4.0 | [
"IID"
] |
- We drop column "toCoupon_GEQ5min" since it has always the same value.
- Anomaly: the data has temporal features but the task is time-invariant.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
ionosphere | 1988 | physics & astronomy | UCI | https://doi.org/10.24432/C5W01B |
mkdir -p local-data-warehouse/ionosphere/ && wget -P local-data-warehouse/ionosphere/ https://archive.ics.uci.edu/static/public/52/ionosphere.zip && unzip local-data-warehouse/ionosphere/ionosphere.zip -d local-data-warehouse/ionosphere/ && rm local-data-warehouse/ionosphere/ionosphere.zip
| @inproceedings{Sigillito1989ClassificationOR,
title={Classification of radar returns from the ionosphere using neural networks},
author={Vincent G. Sigillito and Simon Wing and Larrie V. Hutton and K. L. Baker},
year={1989},
url={https://api.semanticscholar.org/CorpusID:18522381}
}
| Sigillito1989ClassificationOR | CC BY 4.0 | [
"IID"
] |
- We drop "Attribute 2" as it has constant value across samples.
- There is one row duplicate that we keep.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
japanese_credit_screening | 1992 | finance | UCI | https://doi.org/10.24432/C5259N |
We get the credit screening data from UCI.
wget https://archive.ics.uci.edu/static/public/28/japanese+credit+screening.zip && unzip japanese+credit+screening.zip credit.lisp && rm japanese+credit+screening.zip && mkdir -p local-data-warehouse/japanese_credit_screening && mv credit.lisp local-data-warehouse/japanese_... | @misc{Sano1992japaneseCreditScreening,
author = {Sano, Chiharu},
title = {{Japanese Credit Screening}},
year = {1992},
howpublished = {UCI Machine Learning Repository},
note = {{DOI}: https://doi.org/10.24432/C5259N}
}
| Sano1992japaneseCreditScreening | CC BY 4.0 | [
"IID"
] |
We read the data from the .lisp file.
- We treat the `credit_screening` predicate as the target variable, which is binary (1 for positive, 0 for negative).
- We merge female and male indicators into a single `gender` categorical feature.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
kdd_cup_09_appetency | 2008 | business & marketing | Other | https://kdd.org/kdd-cup/view/kdd-cup-2009/Data |
We download the data and the labels from the KDD challenge website:
mkdir -p local-data-warehouse/kdd_cup_09_appetency && wget -P local-data-warehouse/kdd_cup_09_appetency https://kdd.org/cupfiles/KDDCupData/2009/orange_small_train.data.zip && unzip local-data-warehouse/kdd_cup_09_appetency/orange_small_train.data.zi... | @inproceedings{Guyon2009AnalysisOT,
title={Analysis of the KDD Cup 2009: Fast Scoring on a Large Orange Customer Database},
author={Isabelle Guyon and Constantin Aliferis and Greg Cooper and André Elisseeff and Jean-Philippe Pellet and Peter Spirtes and Alexander Statnikov},
booktitle={KDD-Cup 2009 Competition},
... | Guyon2009AnalysisOT | Public Domain | [
"IID"
] |
- We use the small training data from the original data (230 features).
- We use appetency as label.
- We drop 18 empty and 6 zero-variance columns.
- We randomly shuffle the data.
- Anomaly: the feature names and categorical feature values have no semantic meaning.
- A... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
labor_negotiations | 1988 | business & marketing | UCI | https://doi.org/10.24432/C5CP4Q |
We download the dataset from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/labor_negotiations/ && wget -P local-data-warehouse/labor_negotiations/ https://archive.ics.uci.edu/static/public/56/labor+relations.zip && unzip local-data-warehouse/labor_negotiations/labor+relation... | @article{Matwin1988Collective,
title={Collective Bargaining Review},
author={Stan Matwin},
year={1988},
}
| Matwin1988Collective | CC BY 4.0 | [
"IID"
] |
- We merge training and test set and name columns based on provided metainformation.
- We name the feature target "successful_settlement".
- We encode missing values as np.nan instead of "?".
- Anomaly: "standby_pay" and "wage_increase_third_year" features contain only 9 and 15 (out of 57) non-nan val... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
leukemia_allaml | 1999 | medical & healthcare | OpenML | https://www.openml.org/d/1104 |
We download the dataset from OpenML to a predefined folder.
mkdir -p local-data-warehouse/leukemia_allaml/ && wget -P local-data-warehouse/leukemia_allaml/ https://www.openml.org/data/download/53987/leukemia.arff
| @article{Golub1999MolecularCC,
title={Molecular classification of cancer: class discovery and class prediction by gene expression monitoring},
author={Todd R. Golub and Donna K. Slonim and Pablo Tamayo and Christine Huard and Michelle Gaasenbeek and Jill P. Mesirov and Hilary Coller and Mignon L. Loh and James R. D... | Golub1999MolecularCC | Public Domain | [
"IID"
] |
- We use OpenML repository version as the raw data is not available. This dataset contains 7,129 features while the original reference mentions only 6,817 genes. This discrepancy is likely due to some gene filtering based on gene expression level that has not been disclosed in the paper. Hence, we treat OpenML... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
ljubljana_primary_tumor | 1987 | medical & healthcare | UCI | https://doi.org/10.24432/C5WK5Q |
We get the data from UCI.
wget https://archive.ics.uci.edu/static/public/83/primary+tumor.zip && unzip primary+tumor.zip primary-tumor.data && rm primary+tumor.zip && mkdir -p local-data-warehouse/ljubljana_primary_tumor && mv primary-tumor.data local-data-warehouse/ljubljana_primary_tumor/
| @misc{Zwitter1987primarytumor,
author = {Zwitter, M. and Soklic, M.},
title = {{Primary Tumor}},
year = {1987},
howpublished = {UCI Machine Learning Repository},
note = {{DOI}: https://doi.org/10.24432/C5WK5Q}
}
| Zwitter1987primarytumor | CC BY 4.0 | [
"IID"
] |
We start with the data from UCI.
- We encode missing values as np.nan instead of "?".
- We reverse the ordinal encoding of all features, adding the text back to the categories.
- We drop classes with less than 10 samples to have sufficient data per class for a robust benchmark task. In reality, this would call for co... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
lung_cancer | 2001 | medical & healthcare | Other | https://www.pnas.org/doi/10.1073/pnas.191502998#supplementary-materials |
Cannot download automatically!
Click and download yourself: "https://www.pnas.org/doi/suppl/10.1073/pnas.191502998/suppl_file/dataseta_12600gene.xls"
mkdir -p local-data-warehouse/lung_cancer/ && mv dataseta_12600gene.xls local-data-warehouse/lung_cancer/
| @article{bhattacharjee2001classification,
title={Classification of human lung carcinomas by mRNA expression profiling reveals distinct adenocarcinoma subclasses},
author={Bhattacharjee, Arindam and Richards, William G and Staunton, Jane and Li, Cheng and Monti, Stefano and Vasa, Priya and Ladd, Christine and Behesh... | bhattacharjee2001classification | None | [
"IID"
] |
- The original data comes with lung adenocarcinoma and other adenocarcinomas merged into one class already. We keep the same and were not able to find a way to reverse this from the public data.
- We also drop the class "small-cell lung carcinoma" as it only has 6 samples, which is not a meaningful sample size for ben... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
lymphography | 1988 | medical & healthcare | UCI | https://doi.org/10.24432/C54598 |
We get the data from UCI.
wget https://archive.ics.uci.edu/static/public/63/lymphography.zip && unzip lymphography.zip lymphography.data && rm lymphography.zip && mkdir -p local-data-warehouse/lymphography && mv lymphography.data local-data-warehouse/lymphography/
| @misc{Zwitter1988Lymphography,
author = {Zwitter, M. and Soklic, M.},
title = {{Lymphography}},
year = {1988},
howpublished = {UCI Machine Learning Repository},
note = {{DOI}: https://doi.org/10.24432/C54598}
}
| Zwitter1988Lymphography | CC BY 4.0 | [
"IID"
] |
We start with the data from UCI.
- As we only have a too small number of samples for 2 out of the 4 classes, we drop those and keep only the 2 classes with more samples, to have a binary classification task. Thus, we aim to predict given the data if it is metastases or malign lymph.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
marketing_campaign | 2014 | business & marketing | Kaggle | https://www.kaggle.com/datasets/rodsaldanha/arketing-campaign |
We download the data from the Kaggle repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/marketing_campaign/ && wget -O local-data-warehouse/marketing_campaign/marketing_campaign.csv "https://storage.googleapis.com/kagglesdsdata/datasets/178248/1139908/marketing_campaign.csv?X-Goog-Algorithm=... | @misc{uci1990annealing,
title = {Marketing Campaign},
year = {2020},
author = {Saldanha, Rodolfo},
howpublished = {Kaggle},
url = {https://www.kaggle.com/datasets/rodsaldanha/arketing-campaign},
}
| uci1990annealing | CC BY 4.0 | [
"IID"
] |
- We combined train and test data into a single dataset.
- We replaced '?' with 'not_applicable' as described in the metadata.
- We renamed some features to remove '/' characters.
- Anomaly: In the original data, class 4 is in the metadata, but there are no samples in this class.
| null | null | /work/dlclarge1/matusd-data_foundry/data-foundry/local-data-warehouse | dataset-mold-v1 |
mechanisms_of_action | 2021 | biology & life sciences | Kaggle | https://www.kaggle.com/competitions/lish-moa/data |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/mechanisms_of_action/ && cd local-data-warehouse/mechanisms_of_action/ && kaggle competitions download -c lish-moa && cd ../../ && unzip local-data-warehouse/mechanisms_of_action/lish-moa.zip -d local-data-w... | @misc{lish2021moa,
author = {inversion and Jin Paik and Maggie and mrbhbs and Steven Randazzo and tnat1031},
title = {Mechanisms of Action (MoA) Prediction},
year = {2020},
howpublished = {url{https://kaggle.com/competitions/lish-moa}},
note = {Kaggle}
}
| lish2021moa | Competition license | [
"IID"
] |
- We chose the train_features and the train_targets_scored as the full dataset, because for the rest of the data, there is no target
- There are 207 targets, we decide to use the nfkb_inhibitor as the target because of its high prevalence and because it is a central inflammation/immunomodulation target.
| null | null | /work/dlclarge1/matusd-data_foundry/data-foundry/local-data-warehouse | dataset-mold-v1 |
miami_housing | 2016 | finance | Kaggle | https://www.openml.org/d/43093 |
wget https://www.openml.org/data/download/22047757/miami2016.arff && mkdir -p local-data-warehouse/miami_housing && mv miami2016.arff local-data-warehouse/miami_housing/
| @article{bourassa2021big,
title={Big data, accessibility and urban house prices},
author={Bourassa, Steven C and Hoesli, Martin and Merlin, Louis and Renne, John},
journal={Urban Studies},
volume={58},
number={15},
pages={3176--3195},
year={2021},
publisher={SAGE Publications Sage UK: London, England}
}... | bourassa2021big | CC BY-NC-SA | [
"IID",
"Spatial"
] |
- We log scale the target.
- We drop duplicated homes (non-unique identifiers and location) (~1% of the data).
- We drop the ID column "PARCELNO".
- Anomaly: while the original paper describes data with 57k samples, we only have 13k on OpenML or Kaggle.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
mic | 2020 | medical & healthcare | UCI | https://archive.ics.uci.edu/dataset/579/myocardial+infarction+complications |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/mic/ && wget -P local-data-warehouse/mic/ https://archive.ics.uci.edu/static/public/579/myocardial+infarction+complications.zip && unzip local-data-warehouse/mic/myocardial+infarction+complications.zip -d lo... | @article{golovenkin2020trajectories,
title={Trajectories, bifurcations, and pseudo-time in large clinical datasets: applications to myocardial infarction and diabetes data},
author={Golovenkin, Sergey E and Bac, Jonathan and Chervov, Alexander and Mirkes, Evgeny M and Orlova, Yuliya V and Barillot, Emmanuel and Gor... | golovenkin2020trajectories | CC BY 4.0 | [
"IID"
] |
- There are 12 possible targets and four possible time moments to predict the targets for this dataset. We use the categorical target as it summarizes the other possible targets. Furthermore, we use the last possible time point for prediction to utilize all available data.
- We treat "?" as missing values.
- We droppe... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
naticusdroid_android_permissions_dataset | 2021 | technology & internet | UCI | https://archive.ics.uci.edu/dataset/722/naticusdroid+android+permissions+dataset |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/naticusdroid_android_permissions_dataset/ && wget -P local-data-warehouse/naticusdroid_android_permissions_dataset/ https://archive.ics.uci.edu/static/public/722/naticusdroid+android+permissions+dataset.zip ... | @article{mathur2021naticusdroid,
title={NATICUSdroid: A malware detection framework for Android using native and custom permissions},
author={Mathur, Akshay and Podila, Laxmi Mounika and Kulkarni, Keyur and Niyaz, Quamar and Javaid, Ahmad Y},
journal={Journal of Information Security and Applications},
volume={5... | mathur2021naticusdroid | CC BY 4.0 | [
"IID"
] |
- Originally, the data has 75% duplicates and a perfect class balance. The authors never mention duplicates in their paper. Given this large amount of duplicates, we dropped all row-duplicates, keeping only the first entry.
- We drop one duplicated column.
- Anomaly: all features are binary categorical features.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
nci_ovarian_cancer | 2002 | medical & healthcare | GOV Website | https://home.ccr.cancer.gov/ncifdaproteomics/ppatterns.asp |
We download the data to a predefined folder.
mkdir -p local-data-warehouse/nci_ovarian_cancer && wget -P local-data-warehouse/nci_ovarian_cancer https://home.ccr.cancer.gov/ncifdaproteomics/OvarianDataset8-7-02.zip && unzip local-data-warehouse/nci_ovarian_cancer/OvarianDataset8-7-02.zip -d local-data-warehouse/nci_o... | @article{Petricoin2002UseOP,
title={Use of proteomic patterns in serum to identify ovarian cancer},
author={Emanuel F. Petricoin and Ali M. Ardekani and Ben A. Hitt and Peter J. Levine and Vincent A. Fusaro and Seth M. Steinberg and Gordon B. Mills and Charles Simone and David A. Fishman and Elise C. Kohn and Lance... | Petricoin2002UseOP | Public Domain | [
"IID"
] |
- We merge all samples located in the .csv files into one DataFrame.
- We label the samples based on name of the folder they are located in: "Control" translates to healthy patient cases while "Ovarian Cancer" to individuals with an ovarian cancer.
- We drop duplicated 6 columns: Attribute 1514... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
nci_pancreatic_cancer | 2003 | medical & healthcare | GOV Website | https://home.ccr.cancer.gov/ncifdaproteomics/ppatterns.asp |
We download the data to a predefined folder.
mkdir -p local-data-warehouse/nci_pancreatic_cancer && wget -P local-data-warehouse/nci_pancreatic_cancer https://home.ccr.cancer.gov/ncifdaproteomics/CancerCellPanINDataBinned.zip && unzip local-data-warehouse/nci_pancreatic_cancer/CancerCellPanINDataBinned.zip -d local-d... | @article{Hingorani2003PreinvasiveAI,
title={Preinvasive and invasive ductal pancreatic cancer and its early detection in the mouse},
author={Sunil R. Hingorani and Emanuel F. Petricoin and Anirban Maitra and Vinodh Rajapakse and Cynthia King and Michael A. Jacobetz and Sarah Ross and Thomas P. Conrads and Timothy D... | Hingorani2003PreinvasiveAI | Public Domain | [
"IID"
] |
- We merge all samples located in the .csv files into one DataFrame.
- We label the samples based on name of the folder they are located in: "PanINCTL" translates to healthy patient cases while "03-PanIN" to individuals with a pancreatic cancer.
- We shuffle the data since it is concatenated in... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
nci_prostate_cancer | 2002 | medical & healthcare | GOV Website | https://home.ccr.cancer.gov/ncifdaproteomics/ppatterns.asp |
We download the data to a predefined folder.
mkdir -p local-data-warehouse/nci_prostate_cancer && wget -P local-data-warehouse/nci_prostate_cancer https://home.ccr.cancer.gov/ncifdaproteomics/JNCI_Data_7-3-02.zip && unzip local-data-warehouse/nci_prostate_cancer/JNCI_Data_7-3-02.zip -d local-data-warehouse/nci_prosta... | @article{Petricoin2002SerumPP,
title={Serum Proteomic Patterns for Detection of Prostate Cancer},
author={Emanuel F. Petricoin and David K. Ornstein and Cloud P. Paweletz and Ali Ardekani and Paul S. Hackett and Ben A. Hitt and Alfredo Velassco and Christian Trucco and Laura Wiegand and Kamillah Wood and Charles B.... | Petricoin2002SerumPP | Public Domain | [
"IID"
] |
- We merge all samples located in the .csv files into one DataFrame.
- We label the samples based on name of the folder they are located in: "Benign PSA Greater 4" and "No Evidence of Disease PSA Less 1" consitute healthy patient cases while "Prostate Cancer (..)" samples point to individuals with a pr... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
online_shoppers_purchasing_intention_dataset | 2017 | business & marketing | UCI | https://doi.org/10.24432/C5F88Q |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/online_shoppers_purchasing_intention_dataset/ && wget -P local-data-warehouse/online_shoppers_purchasing_intention_dataset/ http://archive.ics.uci.edu/static/public/468/online+shoppers+purchasing+intention+d... | @article{sakar2019pruchasing_intention,
title={Real-time prediction of online shoppers’ purchasing intention using multilayer perceptron and LSTM recurrent neural networks},
author={Sakar, C Okan and Polat, S Olcay and Katircioglu, Mete and Kastro, Yomi},
journal={Neural Computing and Applications},
volume={31}... | sakar2019pruchasing_intention | CC BY 4.0 | [
"IID"
] |
- Anomaly: the data contains time-based features
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
otto_group_product_classification_challenge | 2015 | business & marketing | Kaggle | https://www.kaggle.com/competitions/otto-group-product-classification-challenge |
We use the train.csv from the Kaggle competition.
kaggle competitions download -c otto-group-product-classification-challenge -f train.csv && unzip train.csv.zip && rm train.csv.zip
mkdir -p local-data-warehouse/otto_group_product_classification_challenge && mv train.csv local-data-warehouse/otto_group_product_class... | @misc{Bossan2015OttoGroupProductClassificationChallenge,
author = {Benjamin Bossan and Josef Feigl and Wendy Kan},
title = {Otto Group Product Classification Challenge},
year = {2015},
howpublished = {\url{https://kaggle.com/competitions/otto-group-product-classification-challenge}},
note = {Kaggle compe... | Bossan2015OttoGroupProductClassificationChallenge | Kaggle Competition Rules | [
"IID",
"Anonymized"
] |
We start with the train.csv from Kaggle.
- The data has been anonymized, so feature meanings are unknown.
- Features and data is clean. Top Kaggle experts applied no relevant preprocessing.
- We drop the index column.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
polish_companies_bankruptcy | 2010 | finance | UCI | https://doi.org/10.24432/C5F600 |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/polish_companies_bankruptcy/ && wget -P local-data-warehouse/polish_companies_bankruptcy/ https://archive.ics.uci.edu/static/public/365/polish+companies+bankruptcy+data.zip && unzip local-data-warehouse/poli... | @article{zikeba2016ensemble,
title={Ensemble boosted trees with synthetic features generation in application to bankruptcy prediction},
author={Zi{\k{e}}ba, Maciej and Tomczak, Sebastian K and Tomczak, Jakub M},
journal={Expert systems with applications},
volume={58},
pages={93--101},
year={2016},
publish... | zikeba2016ensemble | CC BY 4.0 | [
"IID"
] |
- We only use data from year 5 (5year.arff), because it is the newest data and the target is bankruptcy status after only 1 year.
- We created semantically meaningful feature names.
- We removed duplicates.
- Anomaly: the data contains a lot of features created by feature engineering.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
porto_seguro | 2017 | insurance | Kaggle | https://www.kaggle.com/c/porto-seguro-safe-driver-prediction |
We use the train.csv from the Kaggle competition.
kaggle competitions download -c porto-seguro-safe-driver-prediction -f train.csv && unzip train.csv.zip && rm train.csv.zip
mkdir -p local-data-warehouse/porto_seguro && mv train.csv local-data-warehouse/porto_seguro/
| @misc{Howard2017PortoSegurosSafeDriverPrediction,
author = {Addison Howard and Adriano Moala and Walter Reade},
title = {Porto Seguro’s Safe Driver Prediction},
year = {2017},
howpublished = {\url{https://kaggle.com/competitions/porto-seguro-safe-driver-prediction}},
note = {Kaggle competition}
}
| Howard2017PortoSegurosSafeDriverPrediction | Kaggle Competition Rules | [
"IID",
"Anonymized"
] |
We start with the train.csv from Kaggle.
- The data has been anonymized, so feature meanings are unknown.
- Following the expert solutions on Kaggle, we drop *calc features.
- In general, the 2nd and 3rd place did a lot of interesting feature engineering for interactions. We do not copy these as the data is clean as ... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
predict_students_dropout_and_academic_success | 2021 | education | UCI | https://doi.org/10.24432/C5MC89 |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/predict_students_dropout_and_academic_success/ && wget -P local-data-warehouse/predict_students_dropout_and_academic_success/ https://archive.ics.uci.edu/static/public/697/predict+students+dropout+and+academ... | @inproceedings{martins2021early,
title={Early prediction of student’s performance in higher education: A case study},
author={Martins, M{\'o}nica V and Tolledo, Daniel and Machado, Jorge and Baptista, Lu{\'\i}s MT and Realinho, Valentim},
booktitle={Trends and Applications in Information Systems and Technologies:... | martins2021early | CC BY 4.0 | [
"IID"
] |
- We renamed the target variable and fixed some typos in the feature names.
- We reversed the ordinal encoding of all categorical features.
- We removed whitespaces and special characters from the feature names.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
prostate_cancer | 2002 | medical & healthcare | OpenML | https://api.openml.org/data/download/22112160/dataset |
We download the dataset from OpenML to a predefined folder:
mkdir -p local-data-warehouse/prostate_cancer/ && wget -P local-data-warehouse/prostate_cancer/ https://api.openml.org/data/download/22112160/dataset
| @article{Singh2002GeneEC,
title={Gene expression correlates of clinical prostate cancer behavior},
author={Dinesh Singh and Phillip G. Febbo and Kenneth Ross and Donald G. Jackson and Judith Manola and Christine Ladd and Pablo Tamayo and Andrew A. Renshaw and Anthony V. D'Amico and Jerome P. Richie and others},
j... | Singh2002GeneEC | Public Domain | [
"IID"
] |
- We use OpenML version of the dataset for binary classification as the URLs from the original reference are outdated.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
pva_revenue_prediction_kddcup98 | 1997 | business & marketing | Other | https://kdd.ics.uci.edu/databases/kddcup98/kddcup98.html |
wget https://kdd.ics.uci.edu/databases/kddcup98/epsilon_mirror/cup98lrn.zip && unzip cup98lrn && rm cup98lrn.zip
wget https://kdd.ics.uci.edu/databases/kddcup98/epsilon_mirror/cup98val.zip && unzip cup98val && rm cup98val.zip && wget https://kdd.ics.uci.edu/databases/kddcup98/epsilon_mirror/valtargt.txt
mkdir -p loca... | @misc{Parsa1998KDDCup1998,
author = {Ismail Parsa},
title = {KDD Cup 1998},
year = {1998},
howpublished = {\url{https://kdd.ics.uci.edu/databases/kddcup98/kddcup98.html}},
note = {Dataset, UCI Machine Learning Repository. DOI: 10.24432/C5401H}
}
| Parsa1998KDDCup1998 | null | [
"IID"
] |
We start with the train and validation data from the KDD Cup 1998 website and combine them.
- The description of the task and data for the KDD Cup (https://kdd.ics.uci.edu/databases/kddcup98/epsilon_mirror/cup98doc.txt) is among the best real-world dataset and task descriptions I have ever seen.
- The task was made I... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
qsar_biodeg | 2013 | biology & life sciences | UCI | https://doi.org/10.24432/C5H60M |
We download the data from Kaggle and uzip it to a predefined folder.
mkdir -p local-data-warehouse/qsar_biodeg/ && wget -P local-data-warehouse/qsar_biodeg/ https://archive.ics.uci.edu/static/public/254/qsar+biodegradation.zip && unzip local-data-warehouse/qsar_biodeg/qsar+biodegradation.zip -d local-data-warehouse/q... | @article{mansouri2013qsar,
title={Quantitative structure--activity relationship models for ready biodegradability of chemicals},
author={Mansouri, Kamel and Ringsted, Tine and Ballabio, Davide and Todeschini, Roberto and Consonni, Viviana},
journal={Journal of chemical information and modeling},
volume={53},
... | mansouri2013qsar | CC BY 4.0 | [
"IID"
] |
- We added semantic meaningful feature names.
- Anomaly: several features are numeric-ordinal in nature but it is unclear if they are categorical features.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
qsar_oral_toxicity | 2019 | biology & life sciences | UCI | https://archive.ics.uci.edu/dataset/508/qsar+oral+toxicity |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/qsar_oral_toxicity && wget -P local-data-warehouse/qsar_oral_toxicity/ https://archive.ics.uci.edu/static/public/508/qsar+oral+toxicity.zip && unzip local-data-warehouse/qsar_oral_toxicity/qsar+oral+toxicit... | @misc{uci2019qsaroral,
title = {QSAR oral toxicity},
year = {2019},
author = {Unknown},
howpublished = {UCI Machine Learning Repository},
url = {https://doi.org/10.24432/C5PS4J},
}
| uci2019qsaroral | CC BY 4.0 | [
"IID"
] |
- We map the original target values to "Yes" and "No" for better readability.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
qsar_tid_11 | 2015 | chemistry & material science | OpenML | https://www.openml.org/d/3050 |
We download the dataset from OpenML to a predefined folder.
mkdir -p local-data-warehouse/qsar_tid_11/ && wget -P local-data-warehouse/qsar_tid_11/ https://www.openml.org/data/download/1674848/11.arff
| @article{olier2018meta,
title={Meta-QSAR: a large-scale application of meta-learning to drug design and discovery},
author={Olier, Ivan and Sadawi, Noureddin and Bickerton, G Richard and Vanschoren, Joaquin and Grosan, Crina and Soldatova, Larisa and King, Ross D},
journal={Machine Learning},
volume={107},
pa... | olier2018meta | Public Domain | [
"IID"
] |
- We drop the "MOLECULE_CHEMBL_ID" column.
- We shuffle the data as it has an original distribution shift.
- Only one row has a negative value (-6.24) and is thus extremely out-of-distribution, which is likely results from the target being log scaled in the original domain. We remove this row as it is unclear if this ... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
santander_customer_transaction_prediction | 2019 | finance | Kaggle | https://www.kaggle.com/c/santander-customer-transaction-prediction |
We use the train.csv from the Kaggle competition.
kaggle competitions download -c santander-customer-transaction-prediction -f train.csv && unzip train.csv.zip && rm train.csv.zip
mkdir -p local-data-warehouse/santander_customer_transaction_prediction && mv train.csv local-data-warehouse/santander_customer_transacti... | @misc{Piedra2019SantanderCustomerTransactionPrediction,
author = {Mercedes Piedra and Sohier Dane and Soraya Jimenez},
title = {Santander Customer Transaction Prediction},
year = {2019},
howpublished = {\url{https://kaggle.com/competitions/santander-customer-transaction-prediction}},
note = {Kaggle compe... | Piedra2019SantanderCustomerTransactionPrediction | Kaggle Competition Rules | [
"IID",
"Anonymized"
] |
We start with the train.csv from Kaggle.
- The data has been anonymized, so feature meanings are unknown.
- The data seems to be generated or created for the competition. Most features are perfectly normally distributed.
- Kaggle experts found various uniqueness-based features and had to filter fake samples from the ... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
sdss_17 | 2022 | physics & astronomy | Kaggle | https://www.kaggle.com/datasets/fedesoriano/stellar-classification-dataset-sdss17 |
We download the data from Kaggle.
mkdir -p local-data-warehouse/sdss_17/ && cd local-data-warehouse/sdss_17 && kaggle datasets download fedesoriano/stellar-classification-dataset-sdss17 && cd ../../ && unzip local-data-warehouse/sdss_17/stellar-classification-dataset-sdss17.zip -d local-data-warehouse/sdss_17 && rm l... | @article{abazajian2009stars,
title={The seventh data release of the Sloan Digital Sky Survey},
author={Abazajian, Kevork N and Adelman-McCarthy, Jennifer K and Ag{\"u}eros, Marcel A and Allam, Sahar S and Prieto, Carlos Allende and An, Deokkeun and Anderson, Kurt SJ and Anderson, Scott F and Annis, James and Bahcal... | abazajian2009stars | Public Domain | [
"IID"
] |
- We renamed the target feature.
- We dropped duplicates based on "obj_ID" to avoid target leakage from subgroups.
- We dropped several (ID-like) meta-features that seem to be not part of the predictive task.
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
smoking_lung_cancer | 2006 | medical & healthcare | GOV Website | https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE4115 |
We download the original gene expression data from the GEO records.
mkdir -p local-data-warehouse/smoking_lung_cancer/ && wget -P local-data-warehouse/smoking_lung_cancer/ https://ftp.ncbi.nlm.nih.gov/geo/series/GSE4nnn/GSE4115/matrix/GSE4115_series_matrix.txt.gz && gzip -d local-data-warehouse/smoking_lung_cancer/GS... | @article{Spira2007AirwayEG,
title={Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer},
author={Avrum Spira and Jennifer Beane and Vishal Shah and Katrina Steiling and Gang Liu and Frank Schembri and Sara Gilman and Yves-Martine Dumas and Paula Calner and Paola Sebast... | Spira2007AirwayEG | Public Domain | [
"IID"
] |
- We start with an extended dataset of 192 patients. The original reference uses 129 samples as "Individuals without final diagnoses as of May 2005 were excluded from this primary dataset".
- We remove 5 samples without a clear label ("suspiction of lung cancer"), yielding a dataset of 187 samples.
... | null | null | /work/dlclarge1/matusd-data_foundry/data-foundry/local-data-warehouse | dataset-mold-v1 |
soybean_large | 1980 | biology & life sciences | UCI | https://doi.org/10.24432/C5JG6Z |
We download the dataset from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/soybean_large/ && wget -P local-data-warehouse/soybean_large/ https://archive.ics.uci.edu/static/public/90/soybean+large.zip && unzip local-data-warehouse/soybean_large/soybean+large.zip -d local-data... | @article{Michalski1980LearningBB,
title={Learning by Being Told and Learning from Examples: An Experimental Comparison of the Two Methods of Knowledge Acquisition in the Context of Developing an Expert System for Soybean Disease Diagnosis},
author={Ryszard S. Michalski and R. L. Chilausky},
journal={International... | Michalski1980LearningBB | CC BY 4.0 | [
"IID"
] |
- We remove "2-4-d-injury" class as it has only a single sample.
- We encode missing values as np.nan instead of "?".
- Anomaly: the dataset contains missing values only for four classes ("phytophthora-rot","diaporthe-pod-&-stem-blight", "cyst-nematode", "herbicide-injury").
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
srbct_prediction | 2001 | medical & healthcare | OpenML | https://www.openml.org/d/45101 |
We download the data from OpenML to a predefined folder.
mkdir -p local-data-warehouse/srbct_prediction/ && wget -P local-data-warehouse/srbct_prediction/ https://api.openml.org/data/download/22112162/dataset
| @article{Khan2001ClassificationAD,
title={Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks},
author={Javed Khan and Jun S. Wei and Markus Ringnér and Lao H. Saal and Marc Ladanyi and Frank Westermann and Frank Berthold and Manfred Schwab and Cristina... | Khan2001ClassificationAD | Public Domain | [
"IID"
] |
- We use the OpenML version of the dataset (training and test excluding 5 non-SRBCT test samples) as the original reference does not mention any official data repostitory.
- We use target feature encoding as in the plsgenomics R package (https://rdrr.io/cran/plsgenomics/man/SRBCT.html).
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
superconductivity | 2018 | physics & astronomy | UCI | https://archive.ics.uci.edu/dataset/464/superconductivty+data |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/superconductivity/ && wget -P local-data-warehouse/superconductivity/ https://archive.ics.uci.edu/static/public/464/superconductivty+data.zip && unzip local-data-warehouse/superconductivity/superconductivty+... | @article{hamidieh2018superconductor,
title={A data-driven statistical model for predicting the critical temperature of a superconductor},
author={Hamidieh, Kam},
journal={Computational Materials Science},
volume={154},
pages={346--354},
year={2018},
publisher={Elsevier}
}
| hamidieh2018superconductor | CC BY 4.0 | [
"IID"
] |
- We combined the two datasets train and unique_m (only target column) into a single dataset.
- Anomaly: the data has a lot of duplicates (29%) when ignoring the target feature
| null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
taiwanese_bankruptcy_prediction | 2009 | finance | UCI | https://doi.org/10.24432/C5004D |
We download the data from the UCI repository and uzip it to a predefined folder.
mkdir -p local-data-warehouse/taiwanese_bankruptcy_prediction/ && wget -P local-data-warehouse/taiwanese_bankruptcy_prediction/ https://archive.ics.uci.edu/static/public/572/taiwanese+bankruptcy+prediction.zip && unzip local-data-warehou... | @article{liang2016financial,
title={Financial ratios and corporate governance indicators in bankruptcy prediction: A comprehensive study},
author={Liang, Deron and Lu, Chia-Chi and Tsai, Chih-Fong and Shih, Guan-An},
journal={European journal of operational research},
volume={252},
number={2},
pages={561--5... | liang2016financial | CC BY 4.0 | [
"IID"
] |
- We map binary Bankrupt column to yes/no
- We rename the features to be shorter and have no special characters.
- We drop the constant column "Net_Income_Flag".
- We drop two duplicated features (same values as other features).
- Anomaly: the paper introducing the dataset has more features, but these do not seem to b... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
thyroid_discordant | 1986 | medical & healthcare | UCI | https://doi.org/10.24432/C5D010 |
We download the dis.data from the UCI files. All other files are corrupted or versions of the same data as far as we know (see the dataset curation sheet for more information).
wget https://archive.ics.uci.edu/static/public/102/thyroid+disease.zip && unzip thyroid+disease.zip dis.data dis.test && rm thyroid+disease.... | @article{quinlan1987simplifying,
title={Simplifying decision trees},
author={Quinlan, J. Ross},
journal={International journal of man-machine studies},
volume={27},
number={3},
pages={221--234},
year={1987},
publisher={Elsevier}
}
| quinlan1987simplifying | CC BY 4.0 | [
"IID"
] |
We start with the .data file from UCI and add columns and encode nan values.
- Note, the data contains NaN indicators by default (e.g. TSH_measured). We keep them.
- We drop/ignore the referral_source columns as it indicates the original of the data and thus might leak information or makes the model learn sub-group r... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
wine_quality | 2009 | chemistry & material science | UCI | https://doi.org/10.24432/C56S3T |
Download from UCI and uzip data to a predefined folder.
mkdir -p local-data-warehouse/wine_quality/ && wget -P local-data-warehouse/wine_quality/ https://archive.ics.uci.edu/static/public/186/wine+quality.zip && unzip local-data-warehouse/wine_quality/wine+quality.zip -d local-data-warehouse/wine_quality/ && rm local-... | @article{Cortez2009ModelingWP,
title={Modeling wine preferences by data mining from physicochemical properties},
author={Paulo Cortez and Antonio Cerdeira and Fernando Almeida and Telmo Matos and Jose Reis},
journal={Decision Support Systems},
year={2009},
volume={47},
number={4},
pages={547-553},
doi={... | Cortez2009ModelingWP | CC BY 4.0 | [
"IID"
] |
- We combine and shuffle the original datasets for red and white wine into a single dataset with an additional column indicating the type of wine (red or white).
- We treat the task as a regression problem, following the original work and because the target is the median wine quality (of at least 3 eva... | null | null | /Users/schaefer.bastian/Programming/ULCO/Autogluon/data-foundry/local-data-warehouse | dataset-mold-v1 |
README.md exists but content is empty.
- Downloads last month
- 517