bgRdZddlZddlmZmZmZmZddlmZm Z m Z m Z m Z ddl mZmZmZmZmZmZmZmZmZmZmZ ddlmZn #e$rYnwxYwddlmZmZmZm Z m!Z!m"Z"m#Z#ddl$m%Z%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+dd l,m-Z-dd l.m/Z/ddl0Z0ddl1Z1ddl2Z2ddl3Z3d Z4d Z5ddZ6GddZ7Gdde Z8Gdde Z9Gdde Z:dS)z3Contains PackIndexFile and PackFile implementationsN) BadObjectAmbiguousObjectNameUnsupportedOperation ParseError)mman LazyMixin unpack_from bin_to_hexbyte_ord) create_pack_object_headerpack_object_header_infois_equal_canonical_shatype_id_to_type_map write_object stream_copy chunk_size delta_types OFS_DELTA REF_DELTAmsb_sizePackIndexFile_sha_to_index)OInfoOStream OPackInfo OPackStream ODeltaStreamODeltaPackInfoODeltaPackStream)DecompressMemMapReaderDeltaApplyReader Sha1Writer NullStreamFlexibleSha1Writer)pack)crc32) NULL_BYTE) PackIndexFilePackFile PackEntitycj||}t|\}}}d}d}|tkrZ|} t || } | dz } | dz} | dzr/t || } | dz } | dz } | dz| dzz} | dz/| }| }n|t kr|dz}|||}n|}||z} |rDt ||dd|} || t|||| fS| t||||| fS|| t|||fS| t||||fS)a& :return: Tuple(abs_data_offset, PackInfo|PackStream) an object of the correct type according to the type_id of the object. If as_stream is True, the object will contain a stream, allowing the data to be read decompressed. :param data: random accessible data containing all required information :parma offset: offset in to the data at which the object information is located :param as_stream: if True, a stream object will be returned that can read the data, otherwise you receive an info object onlyNF) use_regionbufferr rr rr rrrr)cursoroffset as_streamdatatype_id uncomp_sizedata_rela_offsettotal_rela_offset delta_infoic delta_offsetabs_data_offsetstreams [/builddir/build/BUILD/cloudlinux-venv-1.0.7/venv/lib/python3.11/site-packages/gitdb/pack.pypack_object_atrBLs   V $ $ + + - -D-DT-J-J*G[*J)  T!W   Q4x $h <a!!A FA A L(A-!d(;L $h < "  I  ,r1*+<<= -00O ]'->-?-?(@%UU  "Kf$U$UU U"$4VWkS]_e$f$ff f  "Ifg{$K$KK K"N67KQ[$\$\\ \cd}d}|du}d}|r|} |t}|t|z }||} |t| z }|| |rt| |}t|tkrn|} |t| z }|| |rt| |}|||fS)aCopy a stream as read from read function, zip it, and write the result. Count the number of written bytes and return it :param base_crc: if not None, the crc will be the base for all compressed data we consecutively write and generate a crc32 from. If None, no crc will be generated :return: tuple(no bytes read, no bytes written, crc32) crc might be 0 if base_crc was falserN)rlencompressr&flush) readwritezstreambase_crcbrbwwant_crccrcchunk compresseds rAwrite_stream_to_packrRs B Bt#H C Z   c%jj%%e,,  c*oo j  ) C((C u:: # #  J#j//B E*%J$$ C=rCc(eZdZdZdZdZdZdZdS) IndexWriterzUtility to cache index information, allowing to write all information later in one go to the given stream **Note:** currently only writes v2 indices_objsc,t|_dSN)listrUselfs rA__init__zIndexWriter.__init__sVV rCc@|j|||fdS)z&Append one piece of object informationN)rUappend)rZbinsharOr4s rAr]zIndexWriter.appends% 63/00000rCcp|jdt|}|j}|tj|t dtjtd}|jD]+}|t|ddxxdz cc<,tdD]6}||}|t d|||dzxx|z cc<7|t d|d|d d |jD|jD]$}|t d|dd z%t}|jD]V}|d } | d kr*| | d t|zdz } |t d| d zW|D]} |t d| t|dksJ|||d} || | S)zWrite the index file using the given write method :param pack_sha: binary sha over the whole pack that we index :return: sha1 binary sha over all index file contentsc|dS)Nr)os rAz#IndexWriter.write..s adrC)key>L(rrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr,rCc3&K|] }|dV dS)rNra).0ts rA z$IndexWriter.write..s&44A1Q4444444rCi>Qr0Fas_hex)rUsortr$rIr(index_v2_signaturer%index_version_defaultrXr rangejoinr]rEsha) rZpack_sharI sha_writer sha_writetmplistrir<vofsrvs rArIzIndexWriter.writes{ NN+++'.. $  -2333 $t]@AABBBz"" , ,A HQqT!W%% & & &! + & & & &s  A A Id4mm $ $ $ AENNNa NNNN $tWS\**+++  #((4444444555 5 5A Id41 !233 4 4 4 4&& 4 4AA$CZs### 3w<</!3 Id4z!122 3 3 3 3 ' 'C Id4oo & & & &8}}"""" (nnEn** c  rCN)__name__ __module__ __qualname____doc__ __slots__r[r]rIrarCrArTrTsR22I11166666rCrTceZdZdZdZdZdZfdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZdZdZdZdZdevrdZxZSxZS)r(zmA pack index provides offsets into the corresponding pack, allowing to find locations for offsets faster.istOcrlcVt||_dSrW)superr[ _indexpath)rZ indexpath __class__s rAr[zPackIndexFile.__init__s$ #rCcFtj|jd|_dSrW)rforce_map_handle_removal_winr_cursorrYs rAclosezPackIndexFile.closes  )$/::: rCc |dkr(|jdd|_dS|dkr(|jdd|_dS|dkrtj|j|_tjdkrp|jtjkrDtd|j|jtjfzdSdS|j}|dd|j krdpd |_ |j dkr/td |dd}||j ks Jd |zd D]+}t||t|d ||j fz,|dS)N_packfile_checksumrrzsThe index file at %s is too large to fit into a mapped window (%i > %i). This is a limitation of the implementationrlr,rezUnsupported index version: %i)entryr4rvrOz_%s_v%i)rmaprr make_cursorrr1 window_size file_sizeAssertionErrorrr_versionr setattrgetattr _initialize)rZattrmmap version_idfnames rA _set_cache_zPackIndexFile._set_cache_ s ' ' '&*l&6&6&8&8S&AD # # # ) ) )&*l&6&6&8&8&>D # # # Y   +DO<<GGIIDL!!A%%$,*@*@*B*BTEUEWEW*W*W$&[OT\%;%;%=%=t?O?Q?Q^S&STTT&%*W*W<##%%D!"1"X)@@FQL1DM}!!(tQ77: !T]2224SV`4`222 ; X XeWT9t}?U3U%V%VWWWW        rCcdtd|jd|dzzdzS)z!:return: tuple(offset, binsha, 0)z>L20srr rrrZr<s rA _entry_v1zPackIndexFile._entry_v13s/7DL$4$4$6$6q2v FFNNrCcjtd|jd|dzzdS)zsee ``_offset_v2``rerrrrrs rA _offset_v1zPackIndexFile._offset_v17s/4!1!1!3!3TAF]CCAFFrCc`d|dzzdz}|j||dzS)zsee ``_sha_v2``rrrr0rrrZr<bases rA_sha_v1zPackIndexFile._sha_v1;s5q2v"|!!$tby.11rCcdS) unsupportedrrars rA_crc_v1zPackIndexFile._crc_v1@sqrCc~||||||fS)z#:return: tuple(offset, binsha, crc)) _offset_v2_sha_v2_crc_v2rs rA _entry_v2zPackIndexFile._entry_v2Gs/""DLLOOT\\!__EErCctd|j|j|dzzd}|dzr.s/LLU++LLLLLLrC) rarray frombytesrrrrsys byteorderbyteswaptuplertr)rZas` rAoffsetszPackIndexFile.offsetss =A   C  A KK ((**4+>> trCrc"t||SrWrrZrvs rArzPackIndexFile.sha_to_indexs-dC88 8rC)r}r~rrrrrrsr[rrrrrrrrrrrrrrrrrrrrglobals __classcell__)rs@rAr(r(s%% #$$$$$###POOOGGG222 FFF   222 WWWGGG'''+++(((MMM"4111f$wwyy00 9 9 9 9 9 9 91000rCr(cxeZdZdZdZdZdZdZdZdZ dZ d Z dd Z d Z d ZdZdZdZdZdZdZddZdS)r)aA pack is a file written according to the Version 2 for git packs As we currently use memory maps, it could be assumed that the maximum size of packs therefore is 32 bit on 32 bit systems. On 64 bit systems, this should be fine though. **Note:** at some point, this might be implemented using streams as well, or streams are an alternate path in the case memory maps cannot be created for some reason - one clearly doesn't want to read 10GB at once in that case) _packpathr_sizeriKCAPrl r0c||_dSrWr)rZpackpaths rAr[zPackFile.__init__ s !rCcFtj|jd|_dSrW)rrrrrYs rArzPackFile.closes  )$.999 rCc tj|j|_t d|jd\}|_|_||j krtd|zdS)N>LLLrzInvalid pack signature: %i) rrrr1rr rrrpack_signaturer)rZrr7s rArzPackFile._set_cache_s{'77BBDD .9AQAQASASUV-W-W*  d) ) )9GCDD D * )rCTc#K|j}||jz }|p|j}t }||krt ||d\}}t |j|j|j t|j j |j ksJ|||j z |j zz }|r|j d|V||kdSdS)z7Handle the actual iteration of objects within this packTrN)rr footer_sizefirst_object_offsetr#rBrrHrIrrr@_br pack_offsetcompressed_bytes_readseek) rZ start_offsetr5r= content_size cur_offsetnull data_offsetostreams rA _iter_objectszPackFile._iter_objects s L{{}}t'77 !=T%= ||<''#1!Z#F#F K  dj', K K K>%5555 ;)<<@d@d@f@ff fJ  '##A&&&MMM<''''''rCc|jS)z2:return: The amount of objects stored in this pack)rrYs rArz PackFile.size: zrCc|jS)z!:return: the version of this packrrYs rArzPackFile.version>s }rCcX|jS)z :return: read-only data of this pack. It provides random access and usually is a memory map. :note: This method is unsafe as it returns a window into a file which might be larger than than the actual window size)rr1rrYs rAr6z PackFile.dataBs$ |&&((,,...rCc|j|jdz ddS)zE:return: 20 byte sha1 hash on all object sha's contained in this filer0N)rr1rr2rYs rAchecksumzPackFile.checksumJs@|&&t|'='='?'?"'DEELLNNqqqQQrCc|jS)z:return: path to the packfilerrYs rArz PackFile.pathNs ~rCct}|j} t||dd}|||jt kr|j|jz }nnN|S)a :return: list of pack streams which are required to build the object at the given offset. The first entry of the list is the object at offset, the last one is either a full object, or a REF_Delta stream. The latter type needs its reference object to be locked up in an ODB to form a valid delta chain. If the object at offset is no delta, the size of the list is 1. :param offset: specifies the first byte of the object within this packTr,)rXrrBr]r7rrr;)rZr4rr=r s rAcollect_streamszPackFile.collect_streamsUsqff L $Q55a8G JJw   )++ ,w/AA   rCcHt|j|p|jddS)zRetrieve information about the object at the given file-absolute offset :param offset: byte offset :return: OPackInfo instance, the actual type differs depending on the type_id attributeFr,rBrrrZr4s rAinfoz PackFile.infors' dlF,Nd6NPUVVWXYYrCcHt|j|p|jddS)zRetrieve an object at the given file-relative offset as stream along with its information :param offset: byte offset :return: OPackStream instance, the actual type differs depending on the type_id attributeTr,rrs rAr@zPackFile.streamys' dlF,Nd6NPTUUVWXXrCrc0||dS)a :return: iterator yielding OPackStream compatible instances, allowing to access the data in the pack directly. :param start_offset: offset to the first object to iterate. If 0, iteration starts at the very first object in the pack. **Note:** Iterating a pack directly is costly as the datastream has to be decompressed to determine the bounds between the objectsTr5r )rZrs rA stream_iterzPackFile.stream_iters!!,$!???rCN)Tr)r}r~rrrrpack_version_defaultrrr[rrr rrr6rrrrr@rrarCrAr)r)s   >IN K""" E E E4///RRR:ZZZYYY @ @ @ @ @ @rCr)ceZdZdZdZeZeZdZ dZ dZ dZ dZ dd Zd Zd Zd Zd ZdZdZddZdZdZdZdZeddejfdZedejfdZdS)r*zqCombines the PackIndexFile and the PackFile into one, allowing the actual objects to be resolved and iterated)_index_pack _offset_mapctj|\}}|d|z|_|d|z|_dS)zGInitialize ourselves with the path to the respective pack or index filez%s.idxz%s.packN)osrsplitext IndexFileClsr! PackFileClsr")rZpack_or_index_pathbasenameexts rAr[zPackEntity.__init__sT(();<< #''8(;<< %%i(&:;; rCcj|j|jdSrW)r!rr"rYs rArzPackEntity.closes0  rCct|j}t|j|jjz }|s Jdd}t|dkr |d|i}nWt|}t|}t|t||}t|}|||d<||_ dS)NzCannot handle empty indicesr,r) sortedr!rrEr"r6riternextzipdictr#)rZroffsets_sorted last_offset offset_map iter_offsetsiter_offsets_plus_one consecutives rArzPackEntity._set_cache_s   3 3 5 566$*//++,,tz/EE <<<<<~ ~  ! # #(+[9JJ//L$($8$8 ! & ' ' 'l,ABBKk**J.9J~b) *%rCc\|j|}|t||S)z*:return: index for the given sha, or raise)r!rr)rZrvrs rA _sha_to_indexzPackEntity._sha_to_indexs- ((-- =C..  rCc#K|jj}|j}t|jD]}|||||VdS)zPIterate over all objects in our index and yield their OInfo or OStream instencesN)r!rv_objectrtr)rZr5_shar=rs rAr zPackEntity._iter_objectssj{,4;++--.. 9 9E'$$u++y%88 8 8 8 8 9 9rCr.c|dkr||}||j|}|j|}t |jj|\}}}|r|tvr;|j |}t||j |j |j S||} tj| } t#|| j d| S|tvrt%|t&||S||} | dd} t+| \}} t+| |\}} | djtvrt/|dt%|| dj | S)z:return: OInfo or OStream object providing information about the given sha :param index: if not -1, its assumed to be the sha's index in the IndexFilerNir.zCould not resolve delta object)r;r!rvr4r r"rr1r2rr@rtypercollect_streams_at_offsetr!newrrrrHrr7r)rZrvr5rr4r7r8r9 packstreamstreamsdstreambufsrc_size target_sizes rAr=zPackEntity._objects 199&&s++E ;+//%((C##E**1HI[IfIfgmInInIuIuIwIw1x1x..  =k))!Z..v66 sJOZ_jFWXXX 44V< LLL 44V<Y..!&"3Z@@M!%!9!9&:K:S:S:U:U!V!V!%!9!9&:K!L!L~!Z..t{/A/A&/I/IJJFNN6**** "Z..v/@AAFNN6***!.K//*rCc||j||S)ab As ``PackFile.collect_streams``, but takes a sha instead of an offset. Additionally, ref_delta streams will be resolved within this pack. If this is not possible, the stream will be left alone, hence it is adivsed to check for unresolved ref-deltas and resolve them before attempting to construct a delta stream. :param sha: 20 byte sha1 specifying the object whose related streams you want to collect :return: list of streams, first being the actual object delta, the last being a possibly unresolved base object. :raise BadObject:)rAr!r4r;rs rArzPackEntity.collect_streamss7--dk.@.@ASASTWAXAX.Y.YZZZrCNc|}|s:t|ttfst|}t|}t |}|j}d} d} |du} |t dtjtj || dz } | rt} d} |D]} | dz } d}t| j | j }|rt|}nd}||tj|}| j}t%|j|||\}}}|| j ksJ| r| | j|| | t||zz } | |krn| |krt-d|| fz|d }t|d ksJ||| t|z } d}| r| ||}||fS) a Create a new pack by putting all objects obtained by the object_iterator into a pack which is written using the pack_write method. The respective index is produced as well if index_write is not Non. :param object_iter: iterator yielding odb output objects :param pack_write: function to receive strings to write into the pack stream :param indx_write: if not None, the function writes the index file corresponding to the pack. :param object_count: if you can provide the amount of objects in your iteration, this would be the place to put it. Otherwise we have to pre-iterate and store all items into a list to get the number, which uses more memory than necessary. :param zlib_compression: the zlib compression level to use :return: tuple(pack_sha, index_binsha) binary sha over all the contents of the pack and over all contents of the index. If index_write was None, index_binsha will be None **Note:** The destination of the write functions is up to the user. It could be a socket, or a file for instance **Note:** writes only undeltified objectsrNrrr,)rKzKExpected to write %i objects into pack, but received only %i from iteratorsFror0)rrrXrEr$rIr%r)rrrTr r7rr&rT compressobjr@rRrHr]r^rrv)cls object_iter pack_write index_write object_countzlib_compressionobjs pack_writerpwriter|r wants_index actual_countobjrOhdrrJr rLrMrw index_shas rA write_packzPackEntity.write_packsI. %kE4=99 )K((t99L)44 "!-  tFH3X5RT`aabbb r   "MME   C A LC,CKBBC Cjj F3KKK&'788GjG.w|VWWZ[[[KBC>>>> 3 SZc222 3s88b= C|++, < ' ']amo{`||~~ ~ ??%?008}}"""" 8 s8}}  ; Hk::I""rCc<tjdd|\}tjdd|\}fd}fd}||||||\} } tjtjd} tj|| t| dfz} tj|| t| dfz} tj|| tj|| || S)aCreate a new on-disk entity comprised of a properly named pack file and a properly named and corresponding index file. The pack contains all OStream objects contained in object iter. :param base_dir: directory which is to contain the files :return: PackEntity instance initialized with the new pack **Note:** for more information on the other parameters see the write_pack methodr%rc.tj|SrWr%rI)rpack_fds rArcz#PackEntity.create..srx33rCc.tj|SrWr|)rindex_fds rArcz#PackEntity.create..s1 5 5rCz pack-%s.%sidx) tempfilemkstemprxr%rrrur rename)rjrkbase_dirrnro pack_path index_pathrlrm pack_binsha index_binshafmt new_pack_pathnew_index_pathrr}s @@rAcreatezPackEntity.creates&-b&(CC'/GXFF*3333 5555 $'NN; KYegw$x$x! \   Xsj6M6Mv5V/VWW hz+7N7NPU6V0VWW )]+++ *n---s=!!!rC)r.)F)r}r~rrrr(r'r)r(r[rrr;r r=rr@rNrPr%rr_rarrAr classmethodrT Z_BEST_SPEEDrxrrarCrAr*r*s22I !LK<<< &&&2999'='='='=X((('''000///5555n333 222 !!!F [ [ [=A $t7HY#Y#Y#[Y#v8rsZ :9                            ???????   D !!!!!! 5 1]1]1]l%%%%VDDDDDDDDN~9~9~9~9~9I~9~9~9HQ@Q@Q@Q@Q@yQ@Q@Q@lw"w"w"w"w"w"w"w"w"w"sAA  A