3 ÝPfÃ$ã@sØddlmZmZmZddlmZmZmZddlm Z m Z m Z ddl m Z mZmZddlmZmZmZGdd„deƒZGdd „d eƒZGd d „d eƒZGd d „d eƒZGdd„deƒZGdd„deƒZGdd„deƒZdS)é)ÚEUCTW_CHAR_TO_FREQ_ORDERÚEUCTW_TABLE_SIZEÚ EUCTW_TYPICAL_DISTRIBUTION_RATIO)ÚEUCKR_CHAR_TO_FREQ_ORDERÚEUCKR_TABLE_SIZEÚ EUCKR_TYPICAL_DISTRIBUTION_RATIO)ÚGB2312_CHAR_TO_FREQ_ORDERÚGB2312_TABLE_SIZEÚ!GB2312_TYPICAL_DISTRIBUTION_RATIO)ÚBIG5_CHAR_TO_FREQ_ORDERÚBIG5_TABLE_SIZEÚBIG5_TYPICAL_DISTRIBUTION_RATIO)ÚJIS_CHAR_TO_FREQ_ORDERÚJIS_TABLE_SIZEÚJIS_TYPICAL_DISTRIBUTION_RATIOc@sLeZdZdZdZdZdZdd„Zdd„Zd d „Z d d „Z d d„Z dd„Z dS)ÚCharDistributionAnalysisig®Gáz®ï?g{®Gáz„?écCs0d|_d|_d|_d|_d|_d|_|jƒdS)N)Ú_char_to_freq_orderÚ _table_sizeÚtypical_distribution_ratioÚ_doneÚ _total_charsÚ _freq_charsÚreset)Úself©rú&/usr/lib/python3.6/chardistribution.pyÚ__init__.sz!CharDistributionAnalysis.__init__cCsd|_d|_d|_dS)zreset analyser, clear any stateFéN)rrr)rrrrr=szCharDistributionAnalysis.resetcCsX|dkr|j|ƒ}nd}|dkrT|jd7_||jkrTd|j|krT|jd7_dS)z"feed a character with known lengthérriNéÿÿÿÿ)Ú get_orderrrrr)rÚcharZchar_lenÚorderrrrÚfeedFs  zCharDistributionAnalysis.feedcCsT|jdks|j|jkr|jS|j|jkrN|j|j|j|j}||jkrN|S|jS)z(return confidence based on existing datar)rrÚMINIMUM_DATA_THRESHOLDÚSURE_NOrÚSURE_YES)rÚrrrrÚget_confidenceTs   z'CharDistributionAnalysis.get_confidencecCs |j|jkS)N)rÚENOUGH_DATA_THRESHOLD)rrrrÚgot_enough_datadsz(CharDistributionAnalysis.got_enough_datacCsdS)Nrr r)rÚbyte_strrrrr!isz"CharDistributionAnalysis.get_orderN) Ú__name__Ú __module__Ú __qualname__r*r'r&r%rrr$r)r+r!rrrrr(s rcs$eZdZ‡fdd„Zdd„Z‡ZS)ÚEUCTWDistributionAnalysiscs$tt|ƒjƒt|_t|_t|_dS)N) Úsuperr0rrrrrrr)r)Ú __class__rrrrsz"EUCTWDistributionAnalysis.__init__cCs0|d}|dkr(d|d|ddSdSdS)NréÄé^ré¡r r)rr,Ú first_charrrrr!xsz#EUCTWDistributionAnalysis.get_order)r-r.r/rr!Ú __classcell__rr)r2rr0qs r0cs$eZdZ‡fdd„Zdd„Z‡ZS)ÚEUCKRDistributionAnalysiscs$tt|ƒjƒt|_t|_t|_dS)N) r1r8rrrrrrr)r)r2rrr…sz"EUCKRDistributionAnalysis.__init__cCs0|d}|dkr(d|d|ddSdSdS)Nré°r4rr5r r)rr,r6rrrr!‹sz#EUCKRDistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rr8„s r8cs$eZdZ‡fdd„Zdd„Z‡ZS)ÚGB2312DistributionAnalysiscs$tt|ƒjƒt|_t|_t|_dS)N) r1r:rrrr rr r)r)r2rrr˜sz#GB2312DistributionAnalysis.__init__cCs>|d|d}}|dkr6|dkr6d|d|dSdSdS)Nrrr9r5r4r r)rr,r6Ú second_charrrrr!žsz$GB2312DistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rr:—s r:cs$eZdZ‡fdd„Zdd„Z‡ZS)ÚBig5DistributionAnalysiscs$tt|ƒjƒt|_t|_t|_dS)N) r1r<rr rr rr r)r)r2rrr«sz!Big5DistributionAnalysis.__init__cCsX|d|d}}|dkrP|dkr:d|d|ddSd|d|dSndSdS) Nrré¤r5éé?é@r r)rr,r6r;rrrr!±s z"Big5DistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rr<ªs r<cs$eZdZ‡fdd„Zdd„Z‡ZS)ÚSJISDistributionAnalysiscs$tt|ƒjƒt|_t|_t|_dS)N) r1rArrrrrrr)r)r2rrrÁsz!SJISDistributionAnalysis.__init__cCsr|d|d}}|dkr0|dkr0d|d}n&|dkrR|dkrRd|dd}nd S||d }|d krnd }|S) NrrééŸé¼éàéïér@ér r r)rr,r6r;r#rrrr!Çs z"SJISDistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rrAÀs rAcs$eZdZ‡fdd„Zdd„Z‡ZS)ÚEUCJPDistributionAnalysiscs$tt|ƒjƒt|_t|_t|_dS)N) r1rIrrrrrrr)r)r2rrrÚsz"EUCJPDistributionAnalysis.__init__cCs0|d}|dkr(d|d|ddSdSdS)Nré r4r5rr r)rr,r"rrrr!àsz#EUCJPDistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rrIÙs rIN)Z euctwfreqrrrZ euckrfreqrrrZ gb2312freqrr r Zbig5freqr r r ZjisfreqrrrÚobjectrr0r8r:r<rArIrrrrÚsI