MCPcopy Index your code
hub / github.com/Klimabevaegelsen/landbruget.dk

github.com/Klimabevaegelsen/landbruget.dk @main

Chat with this repo
repository ↗ · DeepWiki ↗ · + Follow
7,470 symbols 26,152 edges 1,153 files 4,772 documented · 64%
What it actually does AI analysis from the code graph — generated when you open this
loading…
README

Landbruget.dk

Making Danish agricultural data transparent and universally accessible.

Landbruget.dk organizes data from 18+ Danish government sources into a single, queryable platform. We collect, clean, and publish agricultural, environmental, and regulatory data so that journalists, researchers, and citizens can hold the industry accountable.

Project Structure

landbruget.dk/
├── frontend/               # Next.js 16 — interactive map + data visualization
├── frontend-pesticide/     # Next.js 16 — PFAS/pesticide exposure maps
├── data-explorer/          # Next.js 16 — browser-based SQL queries on Parquet files
├── backend/                # Python data pipelines (medallion architecture)
│   ├── pipelines/          # 12 data pipelines (CHR, unified, climate, etc.)
│   └── common/             # Shared utilities (DuckDB, R2, CRS)
├── supabase/               # PostgreSQL migrations + Edge Functions
├── schema/                 # Data catalog (183 datasets) + relationship docs
├── docs/                   # Pipeline index, data lineage, troubleshooting
├── scripts/                # Utility scripts (worktree setup)
└── .github/                # CI/CD workflows (30+ GitHub Actions)

Tech Stack

Layer Technology Purpose
Frontend Next.js 16, React 19, TypeScript, Tailwind CSS 4 Web applications
Maps MapLibre GL JS, PMTiles Geospatial visualization
Backend Python 3.11+, DuckDB, ibis-framework Data pipelines
Database Supabase (PostgreSQL 15 + PostGIS) Storage + API
Data Storage Cloudflare R2 Raw + processed data
CI/CD GitHub Actions Pipeline orchestration
Deployment Vercel Frontend hosting
Linting oxlint (frontend), ruff (backend) Code quality
Testing Playwright (frontend), pytest (backend) Quality assurance

Quick Start

Prerequisites

  • Node.js 18+
  • Python 3.11
  • uv
  • Supabase CLI

Workspace Setup

./scripts/setup-worktree.sh

This installs frontend dependencies, Playwright browsers, and the shared Python workspace via uv, pinned to Python 3.11 via .python-version, then verifies npm test, npm run lint, and uv run pytest can resolve their local tooling.

Frontend

cd frontend
cp .env.example .env.local    # Configure Supabase credentials
npm ci
npm run dev                   # http://localhost:3000

Data Explorer

cd data-explorer
cp .env.local.example .env.local  # Add Google API key for Gemini
npm install
npm run dev                       # http://localhost:3000

Backend Pipelines

uv sync --python 3.11 --all-packages --group dev

# Run a specific pipeline
cd backend/pipelines/unified_pipeline
uv run python -m unified_pipeline bronze --source cadastral

Data Architecture

Medallion Architecture

All data flows through three layers:

  • Bronze — Raw data preserved exactly as received from sources. No transformations.
  • Silver — Cleaned, validated, and standardized. Type coercion, deduplication, format normalization.
  • Gold — Analysis-ready. Joins across datasets on CVR/CHR/BFE identifiers, derived metrics.

Coordinate Reference System

All geospatial processing uses EPSG:25832 (UTM 32N, meters). Data is transformed to EPSG:4326 (WGS84) only at the final Supabase upload step.

Data Identifiers

All datasets join on one or more of:

Identifier Name Format Purpose
CVR Company Registration 8 digits Links to companies
CHR Central Herd Register 6 digits Links to livestock herds
BFE Cadastral ID Variable Links to land parcels

Data Sources

We collect from 18+ official Danish government sources including:

  • Landbrugsstyrelsen — Field boundaries, crop data, agricultural subsidies
  • Fødevarestyrelsen (FVM) — Livestock registry (CHR), veterinary data, pig movements
  • Miljøstyrelsen — Pesticide database (BMD), environmental company registry (DMA)
  • Geodatastyrelsen — Cadastral data, administrative boundaries
  • Danmarks Statistik — Agricultural statistics
  • DMI — Weather and climate data
  • Arbejdstilsynet — Workplace safety inspections
  • Datafordeleren — Property ownership data
  • GEUS — Borehole pesticide data (Dataverse)

See docs/PIPELINE_INDEX.md for the full pipeline documentation.

Pipelines

Pipeline Purpose Schedule
unified_pipeline 18+ government data sources Weekly (Mon 2 AM UTC)
chr_pipeline Livestock registry + veterinary data Weekly
svineflytning_pipeline Pig movement tracking Weekly (Wed 2 AM UTC)
climate Farm-level CO2e emissions On demand
bmd_scraper Pesticide database Monthly
dma_scraper Environmental company registry Monthly
drive_data_pipeline Google Drive regulatory docs On demand
bbr_buildings Building registry Monthly
arbejdstilsynet_inspections Workplace safety inspections On demand
h3_pfas_exposure_pipeline PFAS exposure mapping Weekly
property_owners_sftp Property ownership data Manual

Development

Running Tests

# Frontend
cd frontend && npm test         # Playwright E2E
cd frontend && npm run lint     # oxlint

# Backend
uv run --all-packages --group dev pytest   # pytest
uv run --all-packages --group dev ruff check backend
uv run --all-packages --group dev ruff format backend

Branch Naming

Format: <type>/<short-description> — e.g. feat/map-view, fix/chr-data-load

Types: feat, fix, docs, refactor, test, chore, ci, perf, build

Commit Messages

<type>(<scope>): <subject>

Examples: feat(frontend): add interactive map view, fix(pipeline): correct CHR transformation

Contributing

  1. Create a branch from main following the naming convention above
  2. Make your changes
  3. Run all tests (npm test + uv run --all-packages --group dev pytest)
  4. Run linters (npm run lint + uv run --all-packages --group dev ruff check backend)
  5. Open a pull request — all PRs require review before merge

License

Code

The source code in this repository is licensed under the MIT License.

Data

The MIT license does not cover the data. This includes both:

  • Upstream data ingested from Danish public-sector sources (Landbrugsstyrelsen, CHR Registry, Geodatastyrelsen, Miljøstyrelsen, Danmarks Statistik, DMI, and others — see docs/PIPELINE_INDEX.md), and
  • Derived datasets we publish to the Cloudflare R2 CDN (JSON, Parquet, PMTiles).

Each dataset retains the licensing terms of its original source — typically Danish government open-data terms or, where applicable, a Creative Commons license. Where the source is not openly licensed, the original copyright and conditions of the issuing public authority apply. Reusing data from this project requires complying with the upstream source's terms.

See /om-os for the project's overall data policy and /kilder for per-source attribution.

The data is provided "as is and as available" — no warranty is given as to completeness, accuracy, or timeliness.

Extension points exported contracts — how you extend this code

BBRBuilding (Interface)
(no doc)
frontend-pesticide/src/types/bbr-data.ts
ScrollamaProps (Interface)
(no doc)
frontend/src/types/react-scrollama.d.ts
MockCompany (Interface)
(no doc)
frontend/tests/utils/test-helpers.ts
DatasetMetadata (Interface)
(no doc)
data-explorer/src/types/index.ts
BBRRawData (Interface)
(no doc)
frontend-pesticide/src/types/bbr-data.ts
StepResponse (Interface)
(no doc)
frontend/src/types/react-scrollama.d.ts
MockField (Interface)
(no doc)
frontend/tests/utils/test-helpers.ts
ColumnSchema (Interface)
(no doc)
data-explorer/src/types/index.ts

Core symbols most depended-on inside this repo

cn
called by 242
frontend/src/lib/utils.ts
list_files
called by 95
backend/pipelines/drive_data_pipeline/utils/storage.py
get_pipeline_logger
called by 90
backend/common/logging_utils.py
query_parquet_direct
called by 52
backend/common/storage/core.py
close
called by 50
backend/pipelines/unified_pipeline/src/unified_pipeline/base/duckdb_processor.py
latest_r2_match
called by 46
backend/pipelines/api_export/exporters/base.py
upload_json
called by 46
backend/common/storage/core.py
create_table_from_storage
called by 45
backend/common/storage/core.py

Shape

Method 3,361
Function 2,854
Class 689
Interface 383
Route 183

Languages

Python79%
TypeScript21%

Modules by API surface

backend/pipelines/unified_pipeline/src/unified_pipeline/gold/nles5_nitrogen_estimation/main.py93 symbols
backend/common/tests/test_crs_utils.py83 symbols
backend/pipelines/api_export/exporters/company_profiles.py64 symbols
backend/common/tests/test_storage_core.py60 symbols
backend/pipelines/unified_pipeline/src/tests/util/test_dawa_api_client.py57 symbols
backend/pipelines/unified_pipeline/src/tests/util/test_storage_access.py54 symbols
backend/pipelines/unified_pipeline/src/tests/util/test_cached_dawa_api_client.py53 symbols
backend/pipelines/chr_pipeline/tests/bronze/test_load_vetstat.py53 symbols
backend/common/tests/test_storage_interface.py52 symbols
backend/scripts/verify_pfas_groundwater_correlations.py51 symbols
backend/scripts/verify_groundwater_correlations.py46 symbols
backend/pipelines/unified_pipeline/tests/gold/test_pmtiles_generator.py45 symbols

Datastores touched

dbDatabase · 1 repos
postgresDatabase · 1 repos

For agents

$ claude mcp add landbruget.dk \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact