Cool Self Hosting Projects Part 2

Now that I’ve had my VPS for a while, here’s some more awesome projects that I’ve decided to self-host.

Grimmory - eBook server#

Grimmory is an ebook server that has two very important features: Native Kobo Sync and KOReader Sync!

Now instead of sideloading my books manually, I load them up in Grimmory, which acts like the Kobo store. It will sync reading status and reading progress, as well any collections or shelves you make.

It also has the ability to offer books on an OPDS (Open Publication Distribution System) feed, which is kind of like an RSS feed for books, and sync reading progress between apps and devices that use KoReader Sync.

The OPDS functionality allows me to browse my Grimmory library and download books directly to the device, which is super convenient.

KOreader sync allows me to read on my XTeink X4, and sync the progress between that and the Kobo (theoretically, the Koreader -> Grimmory sync direction is broken right now, but Kobo -> Grimmory -> Koreader works).

Because Kobo uses an sqlite database for storing reading statistics and progress, it can corrupt easily when connecting to a computer with a book open (since it leaves the database open in case you make highlights or advance in the book). I’ve had my database wiped accidentally multiple times (Installing KOReader has done this 💢), and it’s an annoying hassle. Since Grimmory stores reading progress and books status (finished, reading, unread, etc.), I don’t have to worry about that anymore. I can also still access my Kobo store purchases and use the device as normal, so I really enjoy using this software for sideloading my DRM free books!

Instapaper to OPDS #

I wanted to read my instapaper articles on my XTeink device, since it doesn’t have native Instapaper integration like my Kobo does. However, I didn’t want to go to the hassle of loading the epubs up manually every single time I added an article or fanfic, so I made a tool that will serve up instapaper epubs on an OPDS feed. This feed can then be added to the XTeink, which will allow you to browse and download saved articles from the device.

Readeck - Read-it-later & Archival Service#

Readeck is similar to Pocket and Instapaper, you can save web pages to read later and keep forever.

Why did I install this when I already have Instapaper? Well, it’s because I wanted to use it’s archival features to save fanfics and articles “permanently” (as permanent as my VPS subscription and my backups). This is a paid feature for Instapaper, and I figure it makes more sense to self-host something that I can easily backup and control myself.

Right now, I’m using both Instapaper and Readeck at the same time. Instapaper as a read-it-later tool, Readeck as a save-forever tool. Since I went to painstaking lengths to make my AO3 Email to Instapaper tool, I’m not quitting Instapaper yet lol. Plus, not everything I send to instapaper is something I want to keep forever. I usually delete things when I’m done reading them.

Readeck Kobo Proxy#

Since I’m keeping Instapaper (and it has a native integration on Kobo), I wanted a way to access my Readeck articles on Kobo without replacing the Instapaper endpoint with Readeck. ¹ ²

My solution was to take the OPDS feed served up by Readeck, and create a web proxy accessible from the Kobo browser that allows you to browse your articles, click to download, and have the proxy convert to kepub and serve the file up for download.

I setup a Nickelmenu shortcut that opens up the webpage, but you could also just add the site to your favorites in the kobo browser.

Below is a docker compose example with readeck and the kobo proxy files (Dockerfile, app script, and requirements).

compose.yaml

1
services:
2
  readeck:
3
    image: codeberg.org/readeck/readeck:latest
4
    container_name: readeck
5
    restart: unless-stopped
6
    ports:
7
      - "127.0.0.1:8000:8000" # Secured behind loopback interface for local reverse proxy ingestion
8
    environment:
9
      - READECK_LOG_LEVEL=info
10
      - READECK_LOG_FORMAT=text
11
      - READECK_SERVER_HOST=0.0.0.0
12
      - READECK_SERVER_PORT=8000
13
      - READECK_USE_X_FORWARDED=true
14
    volumes:
15
      - ./data:/readeck
16
    healthcheck:
17
      test: ["CMD", "/bin/readeck", "healthcheck", "-config", "config.toml"]
18
      interval: 30s
19
      timeout: 2s
20
      retries: 3
21

22
  kobo-readeck-proxy:
23
    build:
24
      context: ./kobo-proxy
25
    container_name: kobo-readeck-proxy
26
    ports:
27
      - "5005:5005"
28
    environment:
29
      - READECK_URL=http://readeck:8000  # References internal service block name
30
      - READECK_PASS='your_readeck_api_token_here'
31
    depends_on:
32
      - readeck
33
    restart: unless-stopped

Dockerfile

1
FROM python:3.11-alpine
2

3
# Install curl to pull the binary and runtime dependencies
4
RUN apk add --no-cache curl
5

6
# Download and install the latest stable standalone kepubify binary
7
RUN curl -o /usr/local/bin/kepubify -L "https://github.com/pgaskin/kepubify/releases/download/v4.0.4/kepubify-linux-64bit" \
8
    && chmod +x /usr/local/bin/kepubify
9

10
WORKDIR /app
11

12
COPY requirements.txt .
13
RUN pip install --no-cache-dir -r requirements.txt
14

15
COPY app.py .
16

17
EXPOSE 5005
18

19
CMD ["python", "app.py"]

python script “app(.)py”

1
import os
2
import subprocess
3
import requests
4
import base64
5
import re
6
from bs4 import BeautifulSoup
7
from flask import Flask, Response, render_template_string, request
8

9
app = Flask(__name__)
10

11
READECK_INTERNAL_URL = os.environ.get("READECK_URL", "http://readeck:8000").rstrip('/')
12
READECK_PASS = os.environ.get("READECK_PASS", "").strip().strip("'").strip('"')
13

14
HTML_TEMPLATE = """
15
<!DOCTYPE html>
16
<html>
17
<head>
18
    <meta charset="utf-8">
19
    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">
20
    <title>Readeck Browser</title>
21
    <style>
22
        body { font-family: -apple-system, BlinkMacSystemFont, sans-serif; background: #fff; color: #000; padding: 20px; margin: 0; }
23
        h2 { border-bottom: 3px solid #000; padding-bottom: 10px; font-size: 24px; margin-top: 5px; font-weight: 800; }
24
        .nav-back { display: inline-block; margin-bottom: 15px; font-size: 18px; font-weight: bold; color: #000; text-decoration: none; border: 2px solid #000; padding: 5px 12px; }
25
        ul { list-style: none; padding: 0; margin: 0; }
26
        li { padding: 18px 0; border-bottom: 1px solid #ccc; }
27

28
        .cat-link { font-size: 22px; text-decoration: none; color: #000; font-weight: bold; display: block; }
29
        .cat-link::after { content: "  →"; font-weight: normal; color: #666; }
30

31
        .book-link { font-size: 22px; text-decoration: none; color: #000; font-weight: bold; display: block; line-height: 1.3; }
32
        .book-meta { font-size: 14px; color: #444; margin-top: 6px; letter-spacing: 0.5px; }
33
        .download-tag { display: inline-block; font-size: 12px; font-weight: bold; text-transform: uppercase; border: 1px solid #000; padding: 2px 6px; margin-top: 6px; }
34
    </style>
35
</head>
36
<body>
37
    <h2>{{ feed_title }}</h2>
38

39
    {% if current_path != "opds" %}
40
        <a class="nav-back" href="javascript:history.back()">← Back</a>
41
    {% endif %}
42

43
    <ul>
44
        {% for item in items %}
45
            <li>
46
                {% if item.type == 'catalog' %}
47
                    <a class="cat-link" href="/?path={{ item.target_path | urlencode }}">{{ item.title }}</a>
48
                    {% if item.content %}<div class="book-meta">{{ item.content }}</div>{% endif %}
49
                {% else %}
50
                    <a class="book-link" href="/download?url={{ item.download_url | urlencode }}&title={{ item.title | urlencode }}">{{ item.title }}</a>
51
                    <div class="book-meta">Updated: {{ item.updated }}</div>
52
                    <div class="download-tag">Download KePUB</div>
53
                {% endif %}
54
            </li>
55
        {% else %}
56
            <li>This section is empty.</li>
57
        {% endfor %}
58
    </ul>
59
</body>
60
</html>
61
"""
62

63
@app.route('/')
64
def index():
65
    current_path = request.args.get('path', 'opds').lstrip('/')
66
    target_endpoint = f"{READECK_INTERNAL_URL}/{current_path}"
67

68
    raw_auth_string = f":{READECK_PASS}"
69
    b64_auth_string = base64.b64encode(raw_auth_string.encode('utf-8')).decode('utf-8')
70

71
    headers = {
72
        "Authorization": f"Basic {b64_auth_string}",
73
        "Accept": "application/atom+xml,application/xml"
74
    }
75

76
    try:
77
        response = requests.get(target_endpoint, headers=headers, timeout=15)
78
        if response.status_code != 200:
79
            return f"Readeck returned status code {response.status_code} for path: {current_path}", 500
80
    except requests.exceptions.RequestException as e:
81
        return f"Network link down tracking container nodes: {str(e)}", 500
82

83
    soup = BeautifulSoup(response.content, 'xml')
84

85
    feed_title = soup.find('title').text if soup.find('title') else 'Readeck Catalog'
86

87
    if feed_title.lower() == "opds":
88
        feed_title = "Readeck"
89

90
    items = []
91
    seen_paths = set()
92

93
    # Match raw Base62/random string structures
94
    hash_pattern = re.compile(r'^[a-zA-Z0-9]{20,24}
95

96
requirements.txt
97

98
)
99

100
    # PARSE ENTRIES ONLY
101
    for entry in soup.find_all('entry'):
102
        title = entry.find('title').text.strip() if entry.find('title') else 'Untitled'
103
        content = entry.find('content').text if entry.find('content') else ''
104
        updated = entry.find('updated').text[:10] if entry.find('updated') else 'Unknown'
105

106
        # FIX: ONLY drop the item if the visible DISPLAY TITLE itself is a random hash or "opds"
107
        if hash_pattern.match(title) or title.lower() == 'opds':
108
            continue
109

110
        is_catalog_entry = False
111
        download_url = None
112
        catalog_path = None
113

114
        for link in entry.find_all('link'):
115
            rel = link.get('rel', '')
116
            link_type = link.get('type', '')
117
            href = link.get('href', '')
118

119
            if 'acquisition' in rel and link_type == 'application/epub+zip':
120
                download_url = href
121
                break
122
            elif link_type and 'application/atom+xml' in link_type:
123
                is_catalog_entry = True
124
                catalog_path = href
125

126
        if is_catalog_entry and catalog_path:
127
            clean_path = catalog_path.replace("https://readeck.pomnavi.net", "").replace(READECK_INTERNAL_URL, "").lstrip('/')
128

129
            # Allow hash slugs inside the target URLs, just prevent exact path duplicates
130
            if clean_path not in seen_paths:
131
                seen_paths.add(clean_path)
132
                items.append({
133
                    'type': 'catalog',
134
                    'title': title,
135
                    'target_path': clean_path,
136
                    'content': content[:120] + '...' if len(content) > 120 else content
137
                })
138
        elif download_url:
139
            clean_url = download_url.replace("https://readeck.pomnavi.net", "").replace(READECK_INTERNAL_URL, "")
140
            if clean_url not in seen_paths:
141
                seen_paths.add(clean_url)
142
                items.append({
143
                    'type': 'book',
144
                    'title': title,
145
                    'updated': updated,
146
                    'download_url': clean_url
147
                })
148

149
    return render_template_string(HTML_TEMPLATE, items=items, feed_title=feed_title, current_path=current_path)
150

151
@app.route('/download')
152
def download():
153
    target_url = request.args.get('url')
154
    title = request.args.get('title', 'book')
155

156
    if not target_url:
157
        return "Missing file targets", 400
158

159
    if not target_url.startswith("http"):
160
        target_url = f"{READECK_INTERNAL_URL}/{target_url.lstrip('/')}"
161
    else:
162
        target_url = target_url.replace("https://readeck.pomnavi.net", READECK_INTERNAL_URL)
163

164
    try:
165
        headers = {"Authorization": f"Bearer {READECK_PASS}"}
166
        epub_res = requests.get(target_url, headers=headers, timeout=45)
167
        if epub_res.status_code != 200:
168
            return f"Failed fetching EPUB payload asset: Status {epub_res.status_code}", 500
169
    except requests.exceptions.RequestException as e:
170
        return f"Network connection dropped fetching stream: {str(e)}", 500
171

172
    input_file = "temp_input.epub"
173
    output_file = "temp_output.kepub.epub"
174

175
    with open(input_file, 'wb') as f:
176
        f.write(epub_res.content)
177

178
    subprocess.run(["kepubify", "-o", output_file, input_file])
179

180
    if not os.path.exists(output_file):
181
        if os.path.exists(input_file):
182
            os.remove(input_file)
183
        return "KePUB compilation pipeline dropped execution midway", 500
184

185
    with open(output_file, 'rb') as f:
186
        kepub_data = f.read()
187

188
    os.remove(input_file)
189
    os.remove(output_file)
190

191
    safe_title = "".join([c for c in title if c.isalpha() or c.isdigit() or c in ' .-_']).rstrip()
192

193
    return Response(
194
        kepub_data,
195
        mimetype='application/epub+zip',
196
        headers={'Content-Disposition': f'attachment; filename="{safe_title}.kepub.epub"'}
197
    )
198

199
if __name__ == '__main__':
200
    app.run(host='0.0.0.0', port=5005)

requirements.txt

1
flask==3.0.3
2
requests==2.32.3
3
beautifulsoup4==4.12.3
4
lxml==5.3.0

Comments

Comments are fetched from the Fediverse. You can join the conversation by replying to this post on Hollo. New replies will appear here after the next site rebuild. If you don't have a fedi account, you can send me your comment below.

0 Replies 5 Boosts 1 Likes

No comments yet.

Join the conversation

Grimmory - eBook server#

Instapaper to OPDS#

Readeck - Read-it-later & Archival Service#

Readeck Kobo Proxy#

Footnotes#

Comments

Post a Guest Comment

Instapaper to OPDS #