MCPcopy Create free account
hub / github.com/AhmadIbrahiim/Website-downloader

github.com/AhmadIbrahiim/Website-downloader @1.0.1

Chat with this repo
repository ↗ · DeepWiki ↗ · release 1.0.1 ↗ · + Follow
0 symbols 0 edges 6 files 0 documented · 0% updated 2mo ago★ 4,79616 open issues

Browse by type

Functions 0 Types & classes 0
What it actually does AI analysis from the code graph
loading…
README

Website Downloader 💾

Download the complete source code of any website (including all assets) 🔨.

👉 Live Demo: https://site-dwloader.herokuapp.com/

enter image description here

Description 📒

Website downloader works with wget and archiver to download all websites assets and compress then sends it back to the user through socket channel

Wget params the being used

wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.org

Explanation of the various flags:

  • --mirror – Makes (among other things) the download recursive.
  • --convert-links – convert all the links (also to stuff like CSS stylesheets) to relative, so it will be suitable for offline viewing.
  • --adjust-extension – Adds suitable extensions to filenames (html or css) depending on their content-type.
  • --page-requisites – Download things like CSS style-sheets and images required to properly display the page offline.
  • --no-parent – When recursing do not ascend to the parent directory. It useful for restricting the download to only a portion of the site

How to run it 🤔

  • git clone this repo
  • $ npm install
  • $ npm start

Core symbols most depended-on inside this repo

Shape

Dependencies from manifests, versioned

archiver3.1.1 · 1×
cookie-parser1.4.4 · 1×
debug2.6.9 · 1×
express4.16.1 · 1×
hbs4.0.4 · 1×
http-errors1.6.3 · 1×
jszip3.2.2 · 1×
morgan1.9.1 · 1×
socket.io2.3.0 · 1×
socket.io-client2.3.0 · 1×

For agents

$ claude mcp add Website-downloader \
  -- python -m otcore.mcp_server <graph>

⬇ download graph artifact

Ask about this repo answers extend the page