MCPcopy
hub / github.com/huggingface/datasets / xdirname

Function xdirname

src/datasets/utils/file_utils.py:597–625  ·  view source on GitHub ↗

This function extends os.path.dirname to support the "::" hop separator. It supports both paths and urls. A shorthand, particularly useful where you have multiple hops, is to “chain” the URLs with the special separator "::". This is used to access files inside a zip file over http for

(a)

Source from the content-addressed store, hash-verified

595
596
597def xdirname(a):
598 """
599 This function extends os.path.dirname to support the "::" hop separator. It supports both paths and urls.
600
601 A shorthand, particularly useful where you have multiple hops, is to “chain” the URLs with the special separator "::".
602 This is used to access files inside a zip file over http for example.
603
604 Let's say you have a zip file at https://host.com/archive.zip, and you want to access the file inside the zip file at /folder1/file.txt.
605 Then you can just chain the url this way:
606
607 zip://folder1/file.txt::https://host.com/archive.zip
608
609 The xdirname function allows you to apply the dirname on the first path of the chain.
610
611 Example::
612
613 >>> xdirname("zip://folder1/file.txt::https://host.com/archive.zip")
614 zip://folder1::https://host.com/archive.zip
615 """
616 a, *b = str(a).split("::")
617 if is_local_path(a):
618 a = os.path.dirname(Path(a).as_posix())
619 else:
620 a = posixpath.dirname(a)
621 # if we end up at the root of the protocol, we get for example a = 'http:'
622 # so we have to fix it by adding the '//' that was removed:
623 if a.endswith(":"):
624 a += "//"
625 return "::".join([a] + b)
626
627
628def xexists(urlpath: str, download_config: Optional[DownloadConfig] = None):

Callers 2

test_xdirnameFunction · 0.90
parentMethod · 0.85

Calls 2

is_local_pathFunction · 0.85
splitMethod · 0.80

Tested by 1

test_xdirnameFunction · 0.72