1072 words
5 minutes
Link Fresh - a tool to fight against link rot on my blog

I found a blog post talking about Link rot1 on bubbles.town, and it made me realize that I had run into the same issue as well. Other than using Internet Archive’s Wayback Machine or Archive.today, is there some independent method of archiving pages that I link out to on my blog? I wanted a tool that would ideally work with my Astro blog, rather than a separate self hosted app that I would need to maintain.

Since my blog is hosted on Netlify, I decided to take advantage of Netlify Serverless Functions. I created a script that will save a link that I’ve pasted into the tool as an html page to a folder on my website’s github repo. I can then link to it normally, and it contains a link out to the original post. Instead of fixing posts after the link has rotted, my plan is to save articles that I plan to use as reference in my blog posts, and link to the archived version stored with my site.

Unfortunately this tool fails with a lot of news websites that block scrapers, but I’m just going to rely on Wayback Machine and Archive Today for those articles since I don’t want to fight that arms race, plus I don’t plan on saving everything I ever link out to. Mainly, just other people’s blogs or articles.

You can see an example of a saved page here, which I use in this blog post.

src/pages/admin/linkfresh.astro
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Link Archiver</title>
<style>
body {
font-family: system-ui, -apple-system, sans-serif;
background: #f4f4f9;
color: #333;
display: flex;
justify-content: center;
padding: 2rem 1rem;
}
.card {
background: #ffffff;
max-width: 550px;
width: 100%;
padding: 2rem;
border-radius: 8px;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.08);
}
h1 { margin-top: 0; font-size: 1.5rem; }
.field { margin-bottom: 1.2rem; display: flex; flex-direction: column; gap: 0.4rem; }
label { font-weight: 600; font-size: 0.9rem; }
input {
padding: 0.75rem;
font-size: 1rem;
border: 1px solid #ccc;
border-radius: 4px;
}
button {
padding: 0.75rem;
font-size: 1rem;
font-weight: 600;
background: #0066cc;
color: #fff;
border: none;
border-radius: 4px;
cursor: pointer;
}
button:disabled { background: #888; cursor: not-allowed; }
.status {
margin-top: 1.5rem;
padding: 1rem;
border-radius: 4px;
font-size: 0.95rem;
line-height: 1.4;
}
.hidden { display: none; }
.info { background: #e8f4fd; color: #004085; }
.success { background: #d4edda; color: #155724; }
.error { background: #f8d7da; color: #721c24; }
</style>
</head>
<body>
<div class="card">
<h1>Archive External Link</h1>
<form id="archive-form">
<div class="field">
<label for="url">Target Webpage URL</label>
<input type="url" id="url" name="url" placeholder="https://example.com/blog/post" required />
</div>
<div class="field">
<label for="passcode">Passcode (Optional)</label>
<input type="password" id="passcode" name="passcode" placeholder="Enter ARCHIVE_SECRET if configured" />
</div>
<button type="submit" id="btn">Create Saved Copy</button>
</form>
<div id="status" class="status hidden"></div>
</div>
<script>
const form = document.getElementById('archive-form') as HTMLFormElement;
const urlInput = document.getElementById('url') as HTMLInputElement;
const passInput = document.getElementById('passcode') as HTMLInputElement;
const btn = document.getElementById('btn') as HTMLButtonElement;
const status = document.getElementById('status') as HTMLDivElement;
form.addEventListener('submit', async (e) => {
e.preventDefault();
btn.disabled = true;
btn.textContent = "Archiving & Committing...";
status.className = "status info";
status.textContent = "Fetching page, inlining images, and committing to GitHub...";
try {
const res = await fetch('/api/archive-link', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
url: urlInput.value.trim(),
passcode: passInput.value.trim()
}),
});
const data = await res.json();
if (res.ok && data.success) {
status.className = "status success";
status.innerHTML = `
<strong>Snapshot Saved!</strong><br />
Page location: <a href="${data.path}" target="_blank" rel="noopener">${data.path}</a><br />
<small>Netlify will automatically rebuild your site shortly with this new file.</small>
`;
urlInput.value = '';
} else {
throw new Error(data.error || 'Failed to archive link');
}
} catch (err: any) {
status.className = "status error";
status.textContent = `Error: ${err.message}`;
} finally {
btn.disabled = false;
btn.textContent = "Create Saved Copy";
}
});
</script>
</body>
</html>
import type { Config, Context } from "@netlify/functions";
import * as cheerio from "cheerio";
import { Octokit } from "@octokit/rest";
export default async (req: Request, context: Context) => {
if (req.method !== "POST") {
return new Response(JSON.stringify({ error: "Method not allowed" }), { status: 405 });
}
try {
const { url, passcode } = await req.json();
// Verify simple passphrase protection if set in Netlify env
const expectedSecret = process.env.ARCHIVE_SECRET;
if (expectedSecret && passcode !== expectedSecret) {
return new Response(JSON.stringify({ error: "Unauthorized: Invalid passcode" }), { status: 401 });
}
if (!url) {
return new Response(JSON.stringify({ error: "URL is required" }), { status: 400 });
}
const targetUrl = new URL(url);
// 1. Fetch remote HTML
const res = await fetch(targetUrl.href, {
headers: {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
},
});
if (!res.ok) {
return new Response(JSON.stringify({ error: `Failed to fetch page: ${res.statusText}` }), { status: 400 });
}
const html = await res.text();
const $ = cheerio.load(html);
// Format current date and time (YYYY-MM-DD HH:mm UTC) for placement into the page
const archivedDate = new Date().toISOString().replace('T', ' ').substring(0, 16) + ' UTC';
// 2. Inject sticky top banner with original link context & timestamp
const banner = `
<div style="background:#1a1a1a; color:#ffffff; padding:12px; font-family:sans-serif; text-align:center; font-size:14px; position:sticky; top:0; z-index:999999; border-bottom:2px solid #0066cc;">
Archived Snapshot (${archivedDate}) &bull; <a href="${targetUrl.href}" target="_blank" rel="noopener" style="color:#66b2ff; text-decoration:underline;">View Original Source</a>
</div>
`;
$("body").prepend(banner);
// 3. Convert image sources to inline Base64 data URIs
const imgPromises = $("img")
.map(async (_, el) => {
const src = $(el).attr("src");
if (!src || src.startsWith("data:")) return;
try {
const absoluteSrc = new URL(src, targetUrl.origin).href;
const imgRes = await fetch(absoluteSrc);
if (imgRes.ok) {
const buffer = await imgRes.arrayBuffer();
const base64 = Buffer.from(buffer).toString("base64");
const mimeType = imgRes.headers.get("content-type") || "image/png";
$(el).attr("src", `data:${mimeType};base64,${base64}`);
}
} catch (e) {
// Fallback to absolute remote URL on failure
$(el).attr("src", new URL(src, targetUrl.origin).href);
}
})
.get();
await Promise.all(imgPromises);
// 4. Resolve stylesheet paths to absolute URLs
$('link[rel="stylesheet"]').each((_, el) => {
const href = $(el).attr("href");
if (href) {
$(el).attr("href", new URL(href, targetUrl.origin).href);
}
});
const finalHtml = $.html();
// 5. Generate clean public file path: /links/domain.com-pathname.html
const safeDomain = targetUrl.hostname.replace(/[^a-z0-9.-]/gi, "");
const safePath = targetUrl.pathname.replace(/[^a-z0-9]/gi, "-").replace(/-+/g, "-");
const filePath = `public/links/${safeDomain}${safePath}.html`;
// 6. Commit directly to GitHub via API
const octokit = new Octokit({ auth: process.env.GITHUB_TOKEN });
await octokit.repos.createOrUpdateFileContents({
owner: process.env.GITHUB_OWNER!,
repo: process.env.GITHUB_REPO!,
path: filePath,
message: `archival: add snapshot for ${targetUrl.href} [skip ci]`,
content: Buffer.from(finalHtml).toString("base64"),
branch: "main",
});
return new Response(
JSON.stringify({
success: true,
path: `/links/${safeDomain}${safePath}.html`,
}),
{ status: 200 }
);
} catch (err: any) {
return new Response(JSON.stringify({ error: err.message }), { status: 500 });
}
};
export const config: Config = {
path: "/api/archive-link",
};

Footnotes#

  1. Digital Legacy: Link Rot Mitigation Issues

Comments

Comments are fetched from the Fediverse. You can join the conversation by replying to this post on Hollo. New replies will appear here after the next site rebuild. If you don't have a fedi account, you can send me your comment below.

0 Replies 2 Boosts Likes
No comments yet.

Post a Guest Comment