£Á°èZ¨Ä…–K§‚«“ô4“ÒÙ´dîfUÙÃÅ WKbyÊ¦•êŽ…È®FÒ¿ÊÎóCozá¬S@6{Í:›œêZÌ:Š•_%:¢¾¾~;‘Ã~èŠ©ÊÇí`ÔÑ©úë™µ'5I¿fš×WO%ø9¾«¾DK|€ùÍD”Ýs]nHÕ¶ê×Ó¼ãžªéUWŸÈË%DÒÕ¬ï‘]/Åcx ‰ï2ß]ä6G[]S£ÔÏ¯rs{úëóµmÒï#UQxo·õÞCe]"±/aÙ&Eã4ú9Jé_ÞåëdãöKë)AÞ ¯¹ægƒÛowÐø^d™ý½ßB7áyMä9ÜÖUã !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! 3 �3�X�$�@s�ddlmZmZmZddlmZmZmZddlm Z m Z mZddlm Z mZmZddlmZmZmZGdd�de�ZGdd �d e�ZGd d�de�ZGdd �d e�ZGdd�de�ZGdd�de�ZGdd�de�ZdS)�)�EUCTW_CHAR_TO_FREQ_ORDER�EUCTW_TABLE_SIZE� EUCTW_TYPICAL_DISTRIBUTION_RATIO)�EUCKR_CHAR_TO_FREQ_ORDER�EUCKR_TABLE_SIZE� EUCKR_TYPICAL_DISTRIBUTION_RATIO)�GB2312_CHAR_TO_FREQ_ORDER�GB2312_TABLE_SIZE�!GB2312_TYPICAL_DISTRIBUTION_RATIO)�BIG5_CHAR_TO_FREQ_ORDER�BIG5_TABLE_SIZE�BIG5_TYPICAL_DISTRIBUTION_RATIO)�JIS_CHAR_TO_FREQ_ORDER�JIS_TABLE_SIZE�JIS_TYPICAL_DISTRIBUTION_RATIOc@sLeZdZdZdZdZdZdd�Zdd�Zd d �Z dd�Z d d�Zdd�ZdS)�CharDistributionAnalysisig�G�z��?g{�G�z�?�cCs0d|_d|_d|_d|_d|_d|_|j�dS)N)�_char_to_freq_order�_table_size�typical_distribution_ratio�_done�_total_chars�_freq_chars�reset)�self�r�&/usr/lib/python3.6/chardistribution.py�__init__.sz!CharDistributionAnalysis.__init__cCsd|_d|_d|_dS)zreset analyser, clear any stateF�N)rrr)rrrrr=szCharDistributionAnalysis.resetcCsX|dkr|j|�}nd}|dkrT|jd7_||jkrTd|j|krT|jd7_dS)z"feed a character with known length�rriN��)� get_orderrrrr)r�charZchar_len�orderrrr�feedFs zCharDistributionAnalysis.feedcCsT|jdks|j|jkr|jS|j|jkrN|j|j|j|j}||jkrN|S|jS)z(return confidence based on existing datar)rr�MINIMUM_DATA_THRESHOLD�SURE_NOr�SURE_YES)r�rrrr�get_confidenceTs z'CharDistributionAnalysis.get_confidencecCs|j|jkS)N)r�ENOUGH_DATA_THRESHOLD)rrrr�got_enough_datadsz(CharDistributionAnalysis.got_enough_datacCsdS)Nrr r)r�byte_strrrrr!isz"CharDistributionAnalysis.get_orderN) �__name__� __module__�__qualname__r*r'r&r%rrr$r)r+r!rrrrr(s rcs$eZdZ�fdd�Zdd�Z�ZS)�EUCTWDistributionAnalysiscs$tt|�j�t|_t|_t|_dS)N) �superr0rrrrrrr)r)� __class__rrrrsz"EUCTWDistributionAnalysis.__init__cCs0|d}|dkr(d|d|ddSdSdS)Nr��^r�r r)rr,� first_charrrrr!xsz#EUCTWDistributionAnalysis.get_order)r-r.r/rr!� __classcell__rr)r2rr0qsr0cs$eZdZ�fdd�Zdd�Z�ZS)�EUCKRDistributionAnalysiscs$tt|�j�t|_t|_t|_dS)N) r1r8rrrrrrr)r)r2rrr�sz"EUCKRDistributionAnalysis.__init__cCs0|d}|dkr(d|d|ddSdSdS)Nr�r4rr5r r)rr,r6rrrr!�sz#EUCKRDistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rr8�sr8cs$eZdZ�fdd�Zdd�Z�ZS)�GB2312DistributionAnalysiscs$tt|�j�t|_t|_t|_dS)N) r1r:rrrr rr r)r)r2rrr�sz#GB2312DistributionAnalysis.__init__cCs>|d|d}}|dkr6|dkr6d|d|dSdSdS)Nrrr9r5r4r r)rr,r6�second_charrrrr!�sz$GB2312DistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rr:�sr:cs$eZdZ�fdd�Zdd�Z�ZS)�Big5DistributionAnalysiscs$tt|�j�t|_t|_t|_dS)N) r1r<rrrrrr r)r)r2rrr�sz!Big5DistributionAnalysis.__init__cCsX|d|d}}|dkrP|dkr:d|d|ddSd|d|dSndSdS) Nrr�r5��?�@r r)rr,r6r;rrrr!�sz"Big5DistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rr<�sr<cs$eZdZ�fdd�Zdd�Z�ZS)�SJISDistributionAnalysiscs$tt|�j�t|_t|_t|_dS)N) r1rArrrrrrr)r)r2rrr�sz!SJISDistributionAnalysis.__init__cCsr|d|d}}|dkr0|dkr0d|d}n&|dkrR|dkrRd|dd}ndS||d }|d krnd}|S) Nrr��r@�r r r)rr,r6r;r#rrrr!�sz"SJISDistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rrA�srAcs$eZdZ�fdd�Zdd�Z�ZS)�EUCJPDistributionAnalysiscs$tt|�j�t|_t|_t|_dS)N) r1rIrrrrrrr)r)r2rrr�sz"EUCJPDistributionAnalysis.__init__cCs0|d}|dkr(d|d|ddSdSdS)Nr�r4r5rr r)rr,r"rrrr!�sz#EUCJPDistributionAnalysis.get_order)r-r.r/rr!r7rr)r2rrI�srIN)Z euctwfreqrrrZ euckrfreqrrrZ gb2312freqrr r Zbig5freqrrr Zjisfreqrrr�objectrr0r8r:r<rArIrrrr�sI