MCPcopy
hub / github.com/huggingface/datasets / xsplitext

Function xsplitext

src/datasets/utils/file_utils.py:702–726  ·  view source on GitHub ↗

This function extends os.path.splitext to support the "::" hop separator. It supports both paths and urls. A shorthand, particularly useful where you have multiple hops, is to “chain” the URLs with the special separator "::". This is used to access files inside a zip file over http for

(a)

Source from the content-addressed store, hash-verified

700
701
702def xsplitext(a):
703 """
704 This function extends os.path.splitext to support the "::" hop separator. It supports both paths and urls.
705
706 A shorthand, particularly useful where you have multiple hops, is to “chain” the URLs with the special separator "::".
707 This is used to access files inside a zip file over http for example.
708
709 Let's say you have a zip file at https://host.com/archive.zip, and you want to access the file inside the zip file at /folder1/file.txt.
710 Then you can just chain the url this way:
711
712 zip://folder1/file.txt::https://host.com/archive.zip
713
714 The xsplitext function allows you to apply the splitext on the first path of the chain.
715
716 Example::
717
718 >>> xsplitext("zip://folder1/file.txt::https://host.com/archive.zip")
719 ('zip://folder1/file::https://host.com/archive.zip', '.txt')
720 """
721 a, *b = str(a).split("::")
722 if is_local_path(a):
723 return os.path.splitext(Path(a).as_posix())
724 else:
725 a, ext = posixpath.splitext(a)
726 return "::".join([a] + b), ext
727
728
729def xisfile(path, download_config: Optional[DownloadConfig] = None) -> bool:

Callers 1

test_xsplitextFunction · 0.90

Calls 2

is_local_pathFunction · 0.85
splitMethod · 0.80

Tested by 1

test_xsplitextFunction · 0.72