Skip to content

[Archived.moe] Redirect URL changes (again) #7664

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 15, 2025
Merged

[Archived.moe] Redirect URL changes (again) #7664

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
223fe96
[archivedmoe] redirect URL changes (again)
NecRaul Jun 13, 2025
05081de
Lint with flake8
NecRaul Jun 13, 2025
8b2adeb
[archivedmoe] simplify board URL redirection logic
NecRaul Jun 13, 2025
a7aa18a
[archivedmoe] remove unnecessary logging
NecRaul Jun 13, 2025
3a4e19d
[archivedmoe] Simplify board extraction from url
NecRaul Jun 13, 2025
6668acf
[archivedmoe] Sort boards alphabetically
NecRaul Jun 13, 2025
b245218
[archivedmoe] reword some comments and variable names
mikf Jun 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 31 additions & 7 deletions gallery_dl/extractor/foolfuuka.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,9 @@ def __init__(self, match):
self.remote = self._remote_direct
elif self.category == "archivedmoe":
self.referer = False
self.fixup_timestamp = True
self.fixup_redirect = True
else:
self.fixup_timestamp = False
self.fixup_redirect = False

def items(self):
yield Message.Directory, self.metadata()
Expand Down Expand Up @@ -65,12 +65,36 @@ def remote(self, media):
# '.webm' -> '.web' (#5116)
if url.endswith(".webm"):
url = url[:-1]
elif self.fixup_timestamp:
# trim filename/timestamp to 13 characters (#7652)

elif self.fixup_redirect:
# update redirect domain or filename (#7652)
path, _, filename = url.rpartition("/")
name, _, ext = filename.rpartition(".")
if len(name) > 13:
url = "{}/{}.{}".format(path, name[:13], ext)

# these boards link directly to i.4cdn.org
# -> redirect to warosu or 4plebs instead
board_domains = {
"3" : "warosu.org",
"biz": "warosu.org",
"ck" : "warosu.org",
"diy": "warosu.org",
"fa" : "warosu.org",
"ic" : "warosu.org",
"jp" : "warosu.org",
"lit": "warosu.org",
"sci": "warosu.org",
"tg" : "archive.4plebs.org",
}
board = url.split("/", 4)[3]
if board in board_domains:
domain = board_domains[board]
url = f"https://{domain}/{board}/full_image/{filename}"

# if it's one of these archives, slice the name
elif any(archive in path for archive in (
"b4k.", "desuarchive.", "palanq.")):
name, _, ext = filename.rpartition(".")
if len(name) > 13:
url = f"{path}/{name[:13]}.{ext}"

return url

Expand Down