Skip to content

Commit

Permalink
Archive snapshots of websites locally (#672)
Browse files Browse the repository at this point in the history
* Add basic HTML snapshots

* Implement asset list

* Add snapshot creation tests

* Add deletion tests

* Show file size

* Remove snapshots

* Create new snapshots

* Switch to single-file

* CSS tweak

* Remove auto refresh

* Show delete link when there is no file yet

* Add current date to display name

* Add flag for snapshot support

* Add option for disabling automatic snapshots

* Make snapshots sharable

* Document image variants

* Update README.md

* Add migrations

* Fix tests
  • Loading branch information
sissbruecker committed Apr 1, 2024
1 parent db19069 commit 4280ab4
Show file tree
Hide file tree
Showing 46 changed files with 1,603 additions and 240 deletions.
65 changes: 40 additions & 25 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,16 +33,13 @@ The name comes from:
**Feature Overview:**
- Clean UI optimized for readability
- Organize bookmarks with tags
- Add notes using Markdown
- Read it later functionality
- Share bookmarks with other users
- Bulk editing
- Bulk editing, Markdown notes, read it later functionality
- Share bookmarks with other users or guests
- Automatically provides titles, descriptions and icons of bookmarked websites
- Automatically creates snapshots of bookmarked websites on [the Internet Archive Wayback Machine](https://archive.org/web/)
- Automatically creates snapshots of websites, either as local HTML file or on Internet Archive
- Import and export bookmarks in Netscape HTML format
- Installable as a Progressive Web App (PWA)
- Extensions for [Firefox](https://addons.mozilla.org/firefox/addon/linkding-extension/) and [Chrome](https://chrome.google.com/webstore/detail/linkding-extension/beakmhbijpdhipnjhnclmhgjlddhidpe), as well as a bookmarklet
- Light and dark themes
- SSO support via OIDC or authentication proxies
- REST API for developing 3rd party apps
- Admin panel for user self-service and raw data access
Expand All @@ -62,27 +59,45 @@ The Docker image is compatible with ARM platforms, so it can be run on a Raspber
linkding uses an SQLite database by default.
Alternatively linkding supports PostgreSQL, see the [database options](docs/Options.md#LD_DB_ENGINE) for more information.

<details>

<summary>🧪 Alpine-based image</summary>

The default Docker image (`latest` tag) is based on a slim variant of Debian Linux.
Alternatively, there is an image based on Alpine Linux (`latest-alpine` tag) which has a smaller size, resulting in a smaller download and less disk space required.
The Alpine image is currently about 45 MB in compressed size, compared to about 130 MB for the Debian image.

To use it, replace the `latest` tag with `latest-alpine`, either in the CLI command below when using Docker, or in the `docker-compose.yml` file when using docker-compose.

> [!WARNING]
> The image is currently considered experimental in order to gather feedback and iron out any issues.
> Only use it if you are comfortable running experimental software or want to help out with testing.
> While there should be no issues with creating new installations, there might be issues when migrating existing installations.
> If you plan to migrate your existing installation, make sure to create proper [backups](https://github.com/sissbruecker/linkding/blob/master/docs/backup.md) first.
</details>

### Using Docker

To install linkding using Docker you can just run the [latest image](https://hub.docker.com/repository/docker/sissbruecker/linkding) from Docker Hub:
The Docker image comes in several variants. To use a different image than the default, replace `latest` with the desired tag in the commands below, or in the docker-compose file.

<table>
<thead>
<tr>
<th>Tag</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>latest</code></td>
<td>Provides the basic functionality of linkding</td>
</tr>
<tr>
<td><code>latest-plus</code></td>
<td>
Includes feature for saving HTML snapshots of websites
<ul>
<li>Significantly larger image size as it includes a Chromium installation</li>
<li>Requires more runtime memory to run Chromium</li>
<li>Requires more disk space for storing HTML snapshots</li>
</ul>
</td>
</tr>
<tr>
<td><code>latest-alpine</code></td>
<td><code>latest</code>, but based on Alpine Linux. 🧪 Experimental</td>
</tr>
<tr>
<td><code>latest-plus-alpine</code></td>
<td><code>latest-plus</code>, but based on Alpine Linux. 🧪 Experimental</td>
</tr>
</tbody>
</table>

To install linkding using Docker you can just run the image from [Docker Hub](https://hub.docker.com/repository/docker/sissbruecker/linkding):
```shell
docker run --name linkding -p 9090:9090 -v {host-data-folder}:/etc/linkding/data -d sissbruecker/linkding:latest
```
Expand Down
38 changes: 0 additions & 38 deletions bookmarks/frontend/behaviors/bookmark-details.js

This file was deleted.

6 changes: 3 additions & 3 deletions bookmarks/frontend/behaviors/bookmark-page.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { registerBehavior, swap } from "./index";
import { registerBehavior, swapContent } from "./index";

class BookmarkPage {
constructor(element) {
Expand Down Expand Up @@ -37,8 +37,8 @@ class BookmarkPage {
fetch(`${bookmarksUrl}${query}`).then((response) => response.text()),
fetch(`${tagsUrl}${query}`).then((response) => response.text()),
]).then(([bookmarkListHtml, tagCloudHtml]) => {
swap(this.bookmarkList, bookmarkListHtml);
swap(this.tagCloud, tagCloudHtml);
swapContent(this.bookmarkList, bookmarkListHtml);
swapContent(this.tagCloud, tagCloudHtml);

// Dispatch list updated event
const listElement = this.bookmarkList.querySelector(
Expand Down
54 changes: 54 additions & 0 deletions bookmarks/frontend/behaviors/form.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
import { registerBehavior, swap } from "./index";

class FormBehavior {
constructor(element) {
this.element = element;
element.addEventListener("submit", this.onFormSubmit.bind(this));
}

async onFormSubmit(event) {
event.preventDefault();

const url = this.element.action;
const formData = new FormData(this.element);
if (event.submitter) {
formData.append(event.submitter.name, event.submitter.value);
}

await fetch(url, {
method: "POST",
body: formData,
redirect: "manual", // ignore redirect
});

// Dispatch refresh events
const refreshEvents = this.element.getAttribute("refresh-events");
if (refreshEvents) {
refreshEvents.split(",").forEach((eventName) => {
document.dispatchEvent(new CustomEvent(eventName));
});
}

// Refresh form
await this.refresh();
}

async refresh() {
const refreshUrl = this.element.getAttribute("refresh-url");
const html = await fetch(refreshUrl).then((response) => response.text());
swap(this.element, html);
}
}

class FormAutoSubmitBehavior {
constructor(element) {
this.element = element;
this.element.addEventListener("change", () => {
const form = this.element.closest("form");
form.dispatchEvent(new Event("submit", { cancelable: true }));
});
}
}

registerBehavior("ld-form", FormBehavior);
registerBehavior("ld-form-auto-submit", FormAutoSubmitBehavior);
16 changes: 15 additions & 1 deletion bookmarks/frontend/behaviors/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,14 @@ export function applyBehaviors(container, behaviorNames = null) {

behaviorNames.forEach((behaviorName) => {
const behavior = behaviorRegistry[behaviorName];
const elements = container.querySelectorAll(`[${behaviorName}]`);
const elements = Array.from(
container.querySelectorAll(`[${behaviorName}]`),
);

// Include the container element if it has the behavior
if (container.hasAttribute && container.hasAttribute(behaviorName)) {
elements.push(container);
}

elements.forEach((element) => {
element.__behaviors = element.__behaviors || [];
Expand All @@ -31,6 +38,13 @@ export function applyBehaviors(container, behaviorNames = null) {
}

export function swap(element, html) {
const dom = new DOMParser().parseFromString(html, "text/html");
const newElement = dom.body.firstChild;
element.replaceWith(newElement);
applyBehaviors(newElement);
}

export function swapContent(element, html) {
element.innerHTML = html;
applyBehaviors(element);
}
2 changes: 1 addition & 1 deletion bookmarks/frontend/index.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import "./behaviors/bookmark-details";
import "./behaviors/bookmark-page";
import "./behaviors/bulk-edit";
import "./behaviors/confirm-button";
import "./behaviors/dropdown";
import "./behaviors/form";
import "./behaviors/modal";
import "./behaviors/global-shortcuts";
import "./behaviors/tag-autocomplete";
Expand Down
43 changes: 43 additions & 0 deletions bookmarks/migrations/0030_bookmarkasset.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Generated by Django 5.0.2 on 2024-03-31 08:21

import django.db.models.deletion
from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
("bookmarks", "0029_bookmark_list_actions_toast"),
]

operations = [
migrations.CreateModel(
name="BookmarkAsset",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("date_created", models.DateTimeField(auto_now_add=True)),
("file", models.CharField(blank=True, max_length=2048)),
("file_size", models.IntegerField(null=True)),
("asset_type", models.CharField(max_length=64)),
("content_type", models.CharField(max_length=128)),
("display_name", models.CharField(blank=True, max_length=2048)),
("status", models.CharField(max_length=64)),
("gzip", models.BooleanField(default=False)),
(
"bookmark",
models.ForeignKey(
on_delete=django.db.models.deletion.CASCADE,
to="bookmarks.bookmark",
),
),
],
),
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Generated by Django 5.0.2 on 2024-04-01 10:29

from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
("bookmarks", "0030_bookmarkasset"),
]

operations = [
migrations.AddField(
model_name="userprofile",
name="enable_automatic_html_snapshots",
field=models.BooleanField(default=True),
),
]
34 changes: 34 additions & 0 deletions bookmarks/migrations/0032_html_snapshots_hint_toast.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Generated by Django 5.0.2 on 2024-04-01 12:17

from django.db import migrations
from django.contrib.auth import get_user_model

from bookmarks.models import Toast

User = get_user_model()


def forwards(apps, schema_editor):

for user in User.objects.all():
toast = Toast(
key="html_snapshots_hint",
message="This version adds a new feature for archiving snapshots of websites locally. To use it, you need to switch to a different Docker image. See the installation instructions on GitHub for details.",
owner=user,
)
toast.save()


def reverse(apps, schema_editor):
Toast.objects.filter(key="bookmark_list_actions_hint").delete()


class Migration(migrations.Migration):

dependencies = [
("bookmarks", "0031_userprofile_enable_automatic_html_snapshots"),
]

operations = [
migrations.RunPython(forwards, reverse),
]

0 comments on commit 4280ab4

Please sign in to comment.