I found a blog post talking about Link rot1 on bubbles.town, and it made me realize that I had run into the same issue as well. Other than using Internet Archive’s Wayback Machine or Archive.today, is there some independent method of archiving pages that I link out to on my blog? I wanted a tool that would ideally work with my Astro blog, rather than a separate self hosted app that I would need to maintain.
Since my blog is hosted on Netlify, I decided to take advantage of Netlify Serverless Functions. I created a script that will save a link that I’ve pasted into the tool as an html page to a folder on my website’s github repo. I can then link to it normally, and it contains a link out to the original post. Instead of fixing posts after the link has rotted, my plan is to save articles that I plan to use as reference in my blog posts, and link to the archived version stored with my site.
Unfortunately this tool fails with a lot of news websites that block scrapers, but I’m just going to rely on Wayback Machine and Archive Today for those articles since I don’t want to fight that arms race, plus I don’t plan on saving everything I ever link out to. Mainly, just other people’s blogs or articles.
You can see an example of a saved page here, which I use in this blog post.
EDIT 8/28/2026: Unattributed (see comments below) gave me a pretty good idea about getting past sites with scraper-blocks, and I realized I could probably get around this by just sending the local browser html as a packet using a bookmarklet, just like how Readeck and other read-it-later apps work. So the scripts below have been updated! now it will work on any site :).
Link Fresh page
<html lang="en"> <head> <meta charset="utf-8" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" /> <title>Link Archiver</title> <style> body { font-family: system-ui, -apple-system, sans-serif; background: #f4f4f9; color: #333; display: flex; justify-content: center; padding: 2rem 1rem; } .card { background: #ffffff; max-width: 550px; width: 100%; padding: 2rem; border-radius: 8px; box-shadow: 0 4px 12px rgba(0, 0, 0, 0.08); } h1 { margin-top: 0; font-size: 1.5rem; } .field { margin-bottom: 1.2rem; display: flex; flex-direction: column; gap: 0.4rem; } label { font-weight: 600; font-size: 0.9rem; } input { padding: 0.75rem; font-size: 1rem; border: 1px solid #ccc; border-radius: 4px; } button { padding: 0.75rem; font-size: 1rem; font-weight: 600; background: #0066cc; color: #fff; border: none; border-radius: 4px; cursor: pointer; } button:disabled { background: #888; cursor: not-allowed; } .status { margin-top: 1.5rem; padding: 1rem; border-radius: 4px; font-size: 0.95rem; line-height: 1.4; } .hidden { display: none; } .info { background: #e8f4fd; color: #004085; } .success { background: #d4edda; color: #155724; } .error { background: #f8d7da; color: #721c24; } </style> </head> <body> <div class="card"> <h1>Archive External Link</h1> <form id="archive-form"> <div class="field"> <label for="url">Target Webpage URL</label> <input type="url" id="url" name="url" placeholder="https://example.com/blog/post" required /> </div>
<div class="field"> <label for="passcode">Passcode (Optional)</label> <input type="password" id="passcode" name="passcode" placeholder="Enter ARCHIVE_SECRET if configured" /> </div>
<button type="submit" id="btn">Create Saved Copy</button> </form>
<div id="status" class="status hidden"></div> </div>
<script> const form = document.getElementById('archive-form') as HTMLFormElement; const urlInput = document.getElementById('url') as HTMLInputElement; const passInput = document.getElementById('passcode') as HTMLInputElement; const btn = document.getElementById('btn') as HTMLButtonElement; const status = document.getElementById('status') as HTMLDivElement;
form.addEventListener('submit', async (e) => { e.preventDefault(); btn.disabled = true; btn.textContent = "Archiving & Committing..."; status.className = "status info"; status.textContent = "Fetching page, inlining images, and committing to GitHub...";
try { const res = await fetch('/api/archive-link', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ url: urlInput.value.trim(), passcode: passInput.value.trim() }), });
const data = await res.json();
if (res.ok && data.success) { status.className = "status success"; status.innerHTML = ` <strong>Snapshot Saved!</strong><br /> Page location: <a href="${data.path}" target="_blank" rel="noopener">${data.path}</a><br /> <small>Netlify will automatically rebuild your site shortly with this new file.</small> `; urlInput.value = ''; } else { throw new Error(data.error || 'Failed to archive link'); } } catch (err: any) { status.className = "status error"; status.textContent = `Error: ${err.message}`; } finally { btn.disabled = false; btn.textContent = "Create Saved Copy"; } }); </script> </body></html>Link Fresh Script
import type { Config, Context } from "@netlify/functions";import * as cheerio from "cheerio";import { Octokit } from "@octokit/rest";import sharp from "sharp";
const corsHeaders = { "Access-Control-Allow-Origin": "*", "Access-Control-Allow-Headers": "Content-Type", "Access-Control-Allow-Methods": "POST, OPTIONS", "Content-Type": "application/json",};
export default async (req: Request, context: Context) => { // 1. Handle CORS Preflight OPTIONS requests if (req.method === "OPTIONS") { return new Response(null, { status: 204, headers: corsHeaders }); }
if (req.method !== "POST") { return new Response( JSON.stringify({ error: "Method not allowed" }), { status: 405, headers: corsHeaders } ); }
try { const { url, passcode, html: rawHtml } = await req.json();
const expectedSecret = process.env.ARCHIVE_SECRET; if (expectedSecret && passcode !== expectedSecret) { return new Response( JSON.stringify({ error: "Unauthorized: Invalid passcode" }), { status: 401, headers: corsHeaders } ); }
if (!url) { return new Response( JSON.stringify({ error: "URL is required" }), { status: 400, headers: corsHeaders } ); }
const targetUrl = new URL(url); let html = rawHtml;
// Fetch remote HTML only if local browser DOM wasn't provided if (!html) { const res = await fetch(targetUrl.href, { headers: { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36", }, });
if (!res.ok) { return new Response( JSON.stringify({ error: `Failed to fetch page: ${res.statusText}` }), { status: 400, headers: corsHeaders } ); }
html = await res.text(); }
const $ = cheerio.load(html);
// Prevent embedded scripts from initiating redirects or navigation away from the archived snapshot $("head").prepend(` <meta http-equiv="Content-Security-Policy" content="script-src 'self' 'unsafe-inline' 'unsafe-eval' *; navigate-to 'self';"> `);
const archivedDate = new Date().toISOString().replace("T", " ").substring(0, 16) + " UTC";
// Inject sticky header banner const banner = ` <div style="background:#1a1a1a; color:#ffffff; padding:12px; font-family:sans-serif; text-align:center; font-size:14px; position:sticky; top:0; z-index:999999; border-bottom:2px solid #0066cc;"> Archived Snapshot (${archivedDate}) • <a href="${targetUrl.href}" target="_blank" rel="noopener" style="color:#66b2ff; text-decoration:underline;">View Original Source</a> </div> `; $("body").prepend(banner);
// Compress & convert images to inline Base64 WebP const imgPromises = $("img") .map(async (_, el) => { const src = $(el).attr("src"); if (!src || src.startsWith("data:")) return;
try { const absoluteSrc = new URL(src, targetUrl.origin).href; const imgRes = await fetch(absoluteSrc); if (imgRes.ok) { const buffer = await imgRes.arrayBuffer();
const compressedBuffer = await sharp(Buffer.from(buffer)) .resize({ width: 1000, withoutEnlargement: true }) .webp({ quality: 70 }) .toBuffer();
const base64 = compressedBuffer.toString("base64"); $(el).attr("src", `data:image/webp;base64,${base64}`); } } catch (e) { $(el).attr("src", new URL(src, targetUrl.origin).href); } }) .get();
await Promise.all(imgPromises);
// Inline external stylesheets const cssPromises = $('link[rel="stylesheet"]') .map(async (_, el) => { const href = $(el).attr("href"); if (!href) return;
try { const absoluteHref = new URL(href, targetUrl.origin).href; const cssRes = await fetch(absoluteHref); if (cssRes.ok) { const cssText = await cssRes.text(); const minifiedCss = cssText .replace(/\/\*[\s\S]*?\*\//g, "") .replace(/\s+/g, " ") .replace(/\s*([{}:;,>+~])\s*/g, "$1") .trim();
$(el).replaceWith(`<style>${minifiedCss}</style>`); } else { $(el).attr("href", new URL(href, targetUrl.origin).href); } } catch (e) { $(el).attr("href", new URL(href, targetUrl.origin).href); } }) .get();
await Promise.all(cssPromises);
const finalHtml = $.html();
const safeDomain = targetUrl.hostname.replace(/[^a-z0-9.-]/gi, ""); const safePath = targetUrl.pathname.replace(/[^a-z0-9]/gi, "-").replace(/-+/g, "-"); const filePath = `public/links/${safeDomain}${safePath}.html`;
const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN }); await octokit.repos.createOrUpdateFileContents({ owner: process.env.GITHUB_OWNER!, repo: process.env.GITHUB_REPO!, path: filePath, message: `archival: add snapshot for ${targetUrl.href} [skip ci]`, content: Buffer.from(finalHtml).toString("base64"), branch: "main", });
return new Response( JSON.stringify({ success: true, path: `/links/${safeDomain}${safePath}.html`, }), { status: 200, headers: corsHeaders } ); } catch (err: any) { return new Response( JSON.stringify({ error: err.message }), { status: 500, headers: corsHeaders } ); }};
export const config: Config = { path: "/api/archive-link",};Link Fresh Bookmarklet
javascript:(function(){ const ENDPOINT = "https://yourdomain.com/api/archive-link"; const PASSCODE = "YOUR_ARCHIVE_SECRET";
try { const pageUrl = window.location.href; const pageHtml = document.documentElement.outerHTML;
const statusDiv = document.createElement("div"); Object.assign(statusDiv.style, { position: "fixed", top: "12px", right: "12px", zIndex: "999999999", background: "#1a1a1a", color: "#ffffff", padding: "12px 18px", borderRadius: "6px", fontFamily: "sans-serif", fontSize: "13px", borderLeft: "4px solid #0066cc", boxShadow: "0 4px 12px rgba(0,0,0,0.4)" }); statusDiv.innerText = "Archiving page DOM..."; (document.body || document.documentElement).appendChild(statusDiv);
fetch(ENDPOINT, { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ url: pageUrl, passcode: PASSCODE, html: pageHtml }) }) .then(async (res) => { const data = await res.json(); if (!res.ok) throw new Error(data.error || "Failed to save archive"); return data; }) .then((data) => { statusDiv.style.borderLeftColor = "#28a745"; statusDiv.innerText = "Archived! Saved to " + data.path; console.log("Archive success:", data); }) .catch((err) => { statusDiv.style.borderLeftColor = "#dc3545"; statusDiv.innerText = "Error: " + err.message; console.error("Archive error:", err); }) .finally(() => { setTimeout(() => { if (statusDiv.parentNode) statusDiv.remove(); }, 5000); }); } catch(e) { alert("Bookmarklet failed to execute: " + e.message); }})();
@naviblogs I'm the author of the article you read on Bubbles.
Nice idea with Link Fresh.
Something you might investigate is using RSS feeds for sites that try to block scrapers. Often the RSS feed will have a permalink to an article which can be downloaded and archived. This is how Calibre is able to grab full articles from websites, images and all, and make them into ePub books. (Although some sites will still block you from doing this - like the New York Times.)
@unattributed I'll have to test that out, thanks for replying! Since I'm pasting links manually, I could probably find those permalinks myself.
Honestly my other idea was to use scrapers that rely on what's being displayed in your browser to locally download HTML and upload it myself. I figure this isn't too much work for the occasional links out.