kagglehub

The kagglehub library provides a simple way to interact with Kaggle resources such as datasets, models, notebook outputs in Python.

This library also integrates natively with the Kaggle notebook environment. This means the behavior differs when you download a Kaggle resource with kagglehub in the Kaggle notebook environment:

• In a Kaggle notebook:
  • The resource is automatically attached to your Kaggle notebook.
  • The resource will be shown under the "Input" panel in the Kaggle notebook editor.
  • The resource files are served from the shared Kaggle resources cache (not using the VM's disk).
• Outside a Kaggle notebook:
  • The resource files are downloaded to a local cache folder.

Installation

Install the kagglehub package with pip:

pip install kagglehub

Usage

Authenticate

[!NOTE] kagglehub is authenticated by default when running in a Kaggle notebook.

Authenticating is only needed to access public resources requiring user consent or private resources.

First, you will need a Kaggle account. You can sign up here.

After login, you can download your Kaggle API token at https://www.kaggle.com/settings/api by clicking on the "Generate New Token" button.

You have several options to authenticate. Note that if you use kaggle-api (the kaggle command-line tool) you have already configured authentication and can skip this.

Option 1: kagglehub.login()

This will prompt you to enter your Kaggle API token:

import kagglehub

kagglehub.login()

Option 2: Environment variable

You can also choose to export your Kaggle token to the environment:

export KAGGLE_API_TOKEN=xxxxxxxxxxxxxx # Copied from the settings UI

Option 3: API token file

Store your Kaggle API token obtained from your Kaggle account API tokens settings page in a file at ~/.kaggle/access_token.

Option 4: Google Colab secret

Store your Kaggle API token obtained from your Kaggle account API tokens settings page in a Colab secret named KAGGLE_API_TOKEN.

Instructions on adding secrets in both Colab and Colab Enterprise can be found in this article.

Option 5: Legacy API credentials file

From your Kaggle account API tokens settings page, under "Legacy API Credentials", click on the "Create Legacy API Key" button to generate a kaggle.json file and store it at ~/.kaggle/kaggle.json.

Download Model

The following examples download the answer-equivalence-bem variation of this Kaggle model: https://www.kaggle.com/models/google/bert/tensorFlow2/answer-equivalence-bem

import kagglehub

# Download the latest version.
kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem')

# Download a specific version.
kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem/1')

# Download a single file.
kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem', path='variables/variables.index')

# Download a model or file, even if previously downloaded to cache.
kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem', force_download=True)

# Download to a custom local directory.
kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem', output_dir='./models')

# Overwrite an existing output directory.
kagglehub.model_download('google/bert/tensorFlow2/answer-equivalence-bem', output_dir='./models', force_download=True)

Upload Model

Uploads a new variation (or a new variation's version if it already exists).

import kagglehub

# For example, to upload a new variation to this model:
# - https://www.kaggle.com/models/google/bert/tensorFlow2/answer-equivalence-bem
# 
# You would use the following handle: `google/bert/tensorFlow2/answer-equivalence-bem`
handle = '<KAGGLE_USERNAME>/<MODEL>/<FRAMEWORK>/<VARIATION>'
local_model_dir = 'path/to/local/model/dir'

kagglehub.model_upload(handle, local_model_dir)

# You can also specify some version notes (optional)
kagglehub.model_upload(handle, local_model_dir, version_notes='improved accuracy')

# You can also specify a license (optional)
kagglehub.model_upload(handle, local_model_dir, license_name='Apache 2.0')

# You can also specify a list of patterns for files/dirs to ignore.
# These patterns are combined with `kagglehub.models.DEFAULT_IGNORE_PATTERNS` 
# to determine which files and directories to exclude. 
# To ignore entire directories, include a trailing slash (/) in the pattern.
kagglehub.model_upload(handle, local_model_dir, ignore_patterns=["original/", "*.tmp"])

Load Dataset

Loads a file from a Kaggle Dataset into a python object based on the selected KaggleDatasetAdapter: - KaggleDatasetAdapter.PANDAS → pandas DataFrame (or multiple given certain files/settings) - KaggleDatasetAdapter.HUGGING_FACE→ Hugging Face Dataset - KaggleDatasetAdapter.POLARS → polars LazyFrame or DataFrame (or multiple given certain files/settings)

NOTE: To use these adapters, you must install the optional dependencies (or already have them available in your environment) - KaggleDatasetAdapter.PANDAS → pip install kagglehub[pandas-datasets] - KaggleDatasetAdapter.HUGGING_FACE→ pip install kagglehub[hf-datasets] - KaggleDatasetAdapter.POLARS→ pip install kagglehub[polars-datasets]

KaggleDatasetAdapter.PANDAS

This adapter supports the following file types, which map to a corresponding pandas.read_* method: | File Extension | pandas Method | | ----------------------------------------------- | -------------------------------------------------------------------------------------------------- | | .csv, .tsv[^1] | pandas.read_csv | | .json, .jsonl[^2] | pandas.read_json | | .xml | pandas.read_xml | | .parquet | pandas.read_parquet | | .feather | pandas.read_feather | | .sqlite, .sqlite3, .db, .db3, .s3db, .dl3[^3] | pandas.read_sql_query | | .xls, .xlsx, .xlsm, .xlsb, .odf, .ods, .odt[^4] | pandas.read_excel |

[^1]: For TSV files, \t is automatically supplied for the sep parameter, but may be overridden with pandas_kwargs

[^2]: For JSONL files, True is supplied for the lines parameter

[^3]: For SQLite files, a sql_query must be provided to generate the DataFrame(s)

[^4]: The specific file extension will dictate which optional engine dependency needs to be installed to read the file

dataset_load also supports pandas_kwargs which will be passed as keyword arguments to the pandas.read_* method. Some examples include:

import kagglehub
from kagglehub import KaggleDatasetAdapter

# Load a DataFrame with a specific version of a CSV
df = kagglehub.dataset_load(
    KaggleDatasetAdapter.PANDAS,
    "unsdsn/world-happiness/versions/1",
    "2016.csv",
)

# Load a DataFrame with specific columns from a parquet file
df = kagglehub.dataset_load(
    KaggleDatasetAdapter.PANDAS,
    "robikscube/textocr-text-extraction-from-images-dataset",
    "annot.parquet",
    pandas_kwargs={"columns": ["image_id", "bbox", "points", "area"]}
)

# Load a dictionary of DataFrames from an Excel file where the keys are sheet names 
# and the values are DataFrames for each sheet's data. NOTE: As written, this requires 
# installing the default openpyxl engine.
df_dict = kagglehub.dataset_load(
    KaggleDatasetAdapter.PANDAS,
    "theworldbank/education-statistics",
    "edstats-excel-zip-72-mb-/EdStatsEXCEL.xlsx",
    pandas_kwargs={"sheet_name": None},
)

# Load a DataFrame using an XML file (with the natively available etree parser)
df = dataset_load(
    KaggleDatasetAdapter.PANDAS,
    "parulpandey/covid19-clinical-trials-dataset",
    "COVID-19 CLinical trials studies/COVID-19 CLinical trials studies/NCT00571389.xml",
    pandas_kwargs={"parser": "etree"},
)

# Load a DataFrame by executing a SQL query against a SQLite DB
df = kagglehub.dataset_load(
    KaggleDatasetAdapter.PANDAS,
    "wyattowalsh/basketball",
    "nba.sqlite",
    sql_query="SELECT person_id, player_name FROM draft_history",
)

KaggleDatasetAdapter.HUGGING_FACE

The Hugging Face Dataset provided by this adapater is built exclusively using Dataset.from_pandas. As a result, all of the file type and pandas_kwargs support is the same as KaggleDatasetAdapter.PANDAS. Some important things to note about this:

1. Because Dataset.from_pandas cannot accept a collection of DataFrames, any attempts to load a file with pandas_kwargs that produce a collection of DataFrames will result in a raised exception
2. hf_kwargs may be provided, which will be passed as keyword arguments to Dataset.from_pandas
3. Because the use of pandas is transparent when pandas_kwargs are not needed, we default to False for preserve_index—this can be overridden using hf_kwargs

Some examples include:

import kagglehub
from kagglehub import KaggleDatasetAdapter
# Load a Dataset with a specific version of a CSV, then remove a column
dataset = kagglehub.dataset_load(
    KaggleDatasetAdapter.HUGGING_FACE,
    "unsdsn/world-happiness/versions/1",
    "2016.csv",
)
dataset = dataset.remove_columns('Region')

# Load a Dataset with specific columns from a parquet file, then split into test/train splits
dataset = kagglehub.dataset_load(
    KaggleDatasetAdapter.HUGGING_FACE,
    "robikscube/textocr-text-extraction-from-images-dataset",
    "annot.parquet",
    pandas_kwargs={"columns": ["image_id", "bbox", "points", "area"]}
)
dataset_with_splits = dataset.train_test_split(test_size=0.8, train_size=0.2)

# Load a Dataset by executing a SQL query against a SQLite DB, then rename a column
dataset = kagglehub.dataset_load(
    KaggleDatasetAdapter.HUGGING_FACE,
    "wyattowalsh/basketball",
    "nba.sqlite",
    sql_query="SELECT person_id, player_name FROM draft_history",
)
dataset = dataset.rename_column('season', 'year')

KaggleDatasetAdapter.POLARS

This adapter supports the following file types, which map to a corresponding polars.scan_* or polars.read_* method: | File Extension | polars Method | | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | .csv, .tsv[^1] | polars.scan_csv or polars.read_csv | | .json | polars.read_json | | .jsonl | polars.scan_ndjson or polars.read_ndjson | | .parquet | polars.scan_parquet or polars.read_parquet | | .feather | polars.scan_ipc or polars.read_ipc | | .sqlite, .sqlite3, .db, .db3, .s3db, .dl3[^2] | polars.read_database | | .xls, .xlsx, .xlsm, .xlsb, .odf, .ods, .odt[^3] | polars.read_excel