Add infrastructure to redirect anchors #740
Comments
fricklerhandwerk
added
site
|
What if we simply create the old anchors in the document itself instead of redirecting users? https://www.sphinx-doc.org/en/master/usage/referencing.html#role-ref |
|
This will work for references in text, but not for URLs used in the wild. |
|
Hi @fricklerhandwerk I would like to work on this, but was curious if there's any docs on how |
|
It's now deployed to Netlify thanks to @delroth. That shouldn't be relevant for anchor redirects though, as this can only happen client side. IIUC it also shouldn't matter where the file sits, as you'd have to wire it up in |
|
Ah okay got it - will send a PR soon! |
|
Howdy - I started to dig into this. Initially I was going down the path of using the
I want to say that this is the problem you're trying to solve, but let me know if I'm way off or something! If it is, I'm inclined to test out solution 1 first and see how that goes |
|
Thanks for the digging! Both seem to work at the file level, and indeed we're already using the first. What we're lacking is a way to change heading IDs (the URL fragment part) without disorienting users, which is especially important on large pages. |
|
Hello. By looking at the list of redirects at [nix.dev/blob/master/_redirects](https://github.com/NixOS/nix.dev/blob/master/_redirects) and the script provided I am first thinking something like: const redirects = {
"index.html": {
"old-section": "new-section",
"/tutorials/install-nix": "/install-nix",
// additional mappings
};Now what is the difference between path redirects and anchor redirects since this doesn’t look quite right?
Now I see that there are no anchor links such as |
|
@stablejoy thanks for asking! You described the requirements correctly. The There is now also the first entry in Ideally |
Does that mean we copy the existing functionality of the nix manual client-redirect script and simply add the nix.dev anchors and use that? Something like: Now I am curious about these old anchors. I am looking how to identify and map them. For now I am looking through the nix.dev github repo. |
Exactly. But I'd say no need to overthink it. Having the infrastructure to be able to track future relocations would already be very good. If we happen to find old URLs that went out of date, we can always add redirects after the fact. |
|
Thanks. I'll be working on this, and looking to help out more with the documentation as well. |
|
I have a local working nix.dev repository, and I'm viewing it locally. First I look how to extract the anchor links. By looking at the markdown files I see that the anchor links are not in any way marked. If we look at Example: The page
I also count the lines of the saved Now, I need to manually test a random page such as https://nix.dev/tutorials/packaging-existing-software. I will compare a specific line with the anchor links file and use It outputs nothing which may only mean only the first anchor that is titled same as the page isn't saved in the file. I try now with one of the sub anchors within the page. Now I am not sure why there are three of those so I list all the Yay! so instead of 177 we have 350 anchors now. Let's test it: Now I am curious where are the locations of these six lines of "What do you need?" because for example when I look through the Packaging existing software I only see one "What do you need?" We look for the locations of the files now with: Ok, so this tells us these are all different files. "What do you need?" is indeed a sort of guiding or instructional phrase used at different points within the documentation. What is the next step? We have our anchors link file that has all the titles. Aha, titles. Maybe we need to strip out the Looks good. But when I Now for example the output tells me that
Oh my, I check the original anchor_links.txt and I see that the Additional attributes has four hash symbols before it. So this means the script has to also include up to four levels of hash symbols. Now we see that we have 350 as in the original output of Now how do I turn something like And our install nix page has two anchors actually not just one. So it seems we need more anchors. Let's moditfy the script to include all the lines of our anchors list. The script would be: import json
from collections import defaultdict
# Specify the path to your detailed anchors list file and the output file
input_file = 'detailed_anchors_list_with_fourth_level.txt'
output_file = 'redirects.js'
# Initialize a dictionary to hold the redirects, using defaultdict for nested dictionaries
redirects = defaultdict(dict)
with open(input_file, 'r') as f:
for line in f:
# Assuming the line format is: /path/to/file.md:# Anchor Name
file_path, anchor_text = line.strip().split(':', 1)
# Convert the file path to a simplified web path
web_path = file_path.split('/')[-1].replace('.md', '.html')
# Format the anchor name to a web-friendly ID
anchor_id = anchor_text.strip().lstrip('#').lower().replace(' ', '-').replace('/', '-')
# Assuming each anchor should link to a specific section on the new page
# The new path could be adjusted based on your directory structure or URL scheme
new_path = f"{web_path}#{anchor_id}"
# Map from the anchor ID to the new path
# Note: If your logic for determining the new_path is different, adjust accordingly
redirects[web_path][anchor_id] = new_path
# Convert the redirects dictionary to a JavaScript object notation
redirects_js = f"const redirects = {json.dumps(dict(redirects), indent=2)};"
# Write the JavaScript code to the output file
with open(output_file, 'w') as f:
f.write(redirects_js)
print(f"Redirects have been written to {output_file}")And now our output looks better: How does this look like? This was just an update since it has been for more than a week since I wrote I will work on this. I find the problem super interesting and I am learning a lot. Also thank you very much for opening this issue. Update: I think I have to remove the dashes from the output so I made a new iteration with the python script to strip those down. Now we have: Update: Now we have to append the js logic to these mappings and wire this script into the site. We are instructed to:
We will place our JavaScript file ( We need to see through the file to find where other HTML-related configurations are set, such as Replace the existing redirects object in the nix manual script with our own nix.dev mappings and add it to Let's see how this works: Update: Wonderful. It's not working? First I forgot to add the Now I will test it further and see what's wrong. Update: I have compared the OK, I'm stuck for now, have to explore more, it's like the server and client logic is mixing. I will explore now the nix manual original redirect script to understand better how it works. UPDATE: I look again at the nix manual repository and search for any mention of In nix manual source repository there is also the It doesn't look quite right since Let's see how it works now. It still shows errors but I may be missing something.
I see where is the problem. Oh, our site structure includes multiple pages that themselves contain anchors, and the script did not account for the hierarchical organization of these pages and their subpages. I've updated |
|
@stablejoy thanks for digging into it. Have you considered first building the site and then grepping for I suggest you open a pull request, since your work log here is a lot to chew on but can't be meaningfully reviewed as a code contribution. |
|
I think looking for all elements with an I'm interested in this because I'm looking for a solution to this same problem for the Nixpkgs/NixOS manual as well (both the redirection part and figuring out all possible links that a certain version of the manual could have). @fricklerhandwerk did you think about how a CI check would track changes to anchors? I initially thought about grabbing all possible links from a previous commit and comparing with the commit being checked by the CI, but this is tricky. My next idea was to dump all the possible links in a file that is hosted along with the documentation website, and make the CI compare the links against that:
@stablejoy if/when you make any PRs about this, please ping me as well |
|
Another option is to have a file in the repo which records the previously existing URLs. All of that is a big hack anyway. In my humble opinion, the correct solution is to decouple the URL from the file system location, and encode the history of URLs in a document's metadata. If you do that at HTML element granularity, you get actually persistent URLs, i.e. URIs. You won't get any of that from traditional document processing engines. Notion is a practical example where that part works well. I implemented a half-serious prototype using the module system to show how it could work in combination with source control. |
|
Im on a trip ten more days and I have little connection nor access to a laptop. I have mapped out the links but failed in making the js work. |
Problem
When we move sections from one file to another, there is no way to automatically redirect users to the new location. Using stale anchor links will lead nowhere.
Approaches
The Nix manual uses a client-side redirect script we could copy.
It should be easy to incorporate it into the site configuration.
There doesn't seem to be comparable pre-made tooling for Sphinx, judging from a quick internet search.
Willing to help?
Always.
Priorities
Add 👍 to issues you find important.
The text was updated successfully, but these errors were encountered: