Code
Hub
Workspaces
Following
Trending
Connect
MCP
copy
Create free account
hub
/
github.com/CaledoniaProject/awesome-opensource-security
/ functions
Functions
36 in github.com/CaledoniaProject/awesome-opensource-security
⨍
Functions
36
◇
Types & classes
4
↓ 7 callers
Method
_normalize_url
Normalize URL by removing trailing slash (except for root path).
scripts/main.py:958
↓ 5 callers
Method
_contains_chinese
Check if text contains Chinese characters.
scripts/main.py:1523
↓ 4 callers
Method
parse_files
Parse all .md files in the parent directory, excluding specified files.
scripts/main.py:224
↓ 3 callers
Method
_parse_markdown_link
Parse markdown link and extract title, description, and comment based on URL type. Returns: (title, description, comment) tuple
scripts/main.py:1176
↓ 3 callers
Method
_split_by_separator
Split text by various separator patterns: ' - ', '–', '—', ' – ', ' — ' Returns: (parts, separator_used) tuple
scripts/main.py:1153
↓ 2 callers
Method
_iso_to_timestamp
Convert ISO datetime string to Unix timestamp.
scripts/main.py:104
↓ 2 callers
Method
get_all_data
Retrieve all data from the database.
scripts/main.py:163
↓ 2 callers
Method
process_other_url
Process other URLs by fetching metadata.
scripts/main.py:840
↓ 2 callers
Method
process_url
Process a URL and return metadata based on URL type.
scripts/main.py:915
↓ 2 callers
Method
remove_duplicates
Remove duplicate items while preserving order.
scripts/main.py:214
↓ 1 callers
Method
_build_url_comment_mapping
Build a mapping from URL to Chinese comment by scanning markdown files once.
scripts/main.py:1294
↓ 1 callers
Method
_build_url_description_mapping
Build a mapping from URL to description by scanning markdown files once.
scripts/main.py:1338
↓ 1 callers
Method
_build_url_title_mapping
Build a mapping from URL to title by scanning markdown files once.
scripts/main.py:1382
↓ 1 callers
Method
delete_failed_entries
Delete all failed entries (status = 0) from the database.
scripts/main.py:187
↓ 1 callers
Method
dump_data
Dump data from the database that exists in markdown files and has succeeded.
scripts/main.py:1037
↓ 1 callers
Method
dump_data_minimized
Dump data in JSON format that can be decoded as a list of Go structs.
scripts/main.py:1070
↓ 1 callers
Method
get_existing_entry
Get existing entry for a URL.
scripts/main.py:116
↓ 1 callers
Method
get_failed_entries
Retrieve only failed entries (status = 0) from the database.
scripts/main.py:175
↓ 1 callers
Method
init_database
Initialize the database with the required table structure.
scripts/main.py:51
↓ 1 callers
Method
insert_or_update
Insert or update a record in the database.
scripts/main.py:75
↓ 1 callers
Function
main
()
scripts/main.py:1546
↓ 1 callers
Method
process_crates_url
Process Crates.io URLs.
scripts/main.py:587
↓ 1 callers
Method
process_gist_url
Process GitHub Gist URLs.
scripts/main.py:355
↓ 1 callers
Method
process_gitee_url
Process Gitee repository URLs.
scripts/main.py:650
↓ 1 callers
Method
process_github_url
Process GitHub repository URLs, user/organization URLs, and other GitHub URLs.
scripts/main.py:407
↓ 1 callers
Method
process_gitlab_url
Process GitLab repository URLs.
scripts/main.py:524
↓ 1 callers
Method
process_pdf_url
(self, url)
scripts/main.py:733
↓ 1 callers
Method
scan_comments
Scan all markdown files and extract Chinese comments matching pattern ` - .* - (.*)`.
scripts/main.py:1475
↓ 1 callers
Method
show_failed_entries
Show only failed entries from the database that exist in markdown files.
scripts/main.py:1426
↓ 1 callers
Method
test_url
Test URL parsing and return the result.
scripts/main.py:1452
↓ 1 callers
Method
update_database
Update the local SQLite database with data from markdown files.
scripts/main.py:968
Method
__init__
(self, db_path="data.db")
scripts/main.py:47
Method
__init__
(self, base_path)
scripts/main.py:211
Method
__init__
(self, github_token=None, proxy=None)
scripts/main.py:317
Method
__init__
(self, github_token=None, proxy=None)
scripts/main.py:947
Method
should_update_entry
Determine if an entry should be updated based on age and status.
scripts/main.py:128