Your IP : 216.73.216.213
3
� f�% � @ s� d Z ddlZddlZddlZddlZddlmZ ddlmZ ddl Z
yddl m Z W n e
k
rt ddlm Z Y nX ejdejejB �jZejdkZG dd � d �ZG d
d � d ej�ZG d d
� d
e�Zdd� Zdd� ZG dd� dej �Z dS )a�
Try to detect suspicious constructs, resembling markup
that has leaked into the final output.
Suspicious lines are reported in a comma-separated-file,
``suspicious.csv``, located in the output directory.
The file is utf-8 encoded, and each line contains four fields:
* document name (normalized)
* line number in the source document
* problematic text
* complete line showing the problematic text in context
It is common to find many false positives. To avoid reporting them
again and again, they may be added to the ``ignored.csv`` file
(located in the configuration directory). The file has the same
format as ``suspicious.csv`` with a few differences:
- each line defines a rule; if the rule matches, the issue
is ignored.
- line number may be empty (that is, nothing between the
commas: ",,"). In this case, line numbers are ignored (the
rule matches anywhere in the file).
- the last field does not have to be a complete line; some
surrounding text (never more than a line) is enough for
context.
Rules are processed sequentially. A rule matches when:
* document names are the same
* problematic texts are the same
* line numbers are close to each other (5 lines up or down)
* the rule text is completely contained into the source line
The simplest way to create the ignored.csv file is by copying
undesired entries from suspicious.csv (possibly trimming the last
field.)
Copyright 2009 Gabriel A. Genellina
� N)�nodes)�Builder)� getLoggerz�
::(?=[^=])| # two :: (but NOT ::=)
:[a-zA-Z][a-zA-Z0-9]+| # :foo
`| # ` (seldom used by itself)
(?<!\.)\.\.[ \t]*\w+: # .. foo: (but NOT ... else:)
� c @ s e Zd Zdd� Zdd� ZdS )�Rulec C s" || _ || _|| _|| _d| _dS )zA rule for ignoring issuesFN)�docname�lineno�issue�line�used)�selfr r r r
� r
�D/opt/alt/python36/lib64/python3.6/Doc/tools/extensions/suspicious.py�__init__F s
z
Rule.__init__c C s
dj | �S )Nz {0.docname},,{0.issue},{0.line})�format)r r
r
r �__repr__P s z
Rule.__repr__N)�__name__�
__module__� __qualname__r r r
r
r
r r E s
r c @ s e Zd ZdZdZdS )�dialectz+Our dialect: uses only linefeed as newline.�
N)r r r �__doc__Zlineterminatorr
r
r
r r U s r c @ sv e Zd ZdZdZed �Zdd� Zdd� Zddd �Z d
d � Z
d d
� Z dd� Z dd� Z
dd� Zdd� Zdd� Zdd� ZdS )� CheckSuspiciousMarkupBuilderzK
Checks for possibly invalid markup that may leak into the output.
Z
suspiciousc C sD t jj| jd�| _t| jd�j� | jt jjt jjt �dd�� d S )Nzsuspicious.csv�wz..zsusp-ignored.csv)
�os�path�joinZoutdir�
log_file_name�open�close�
load_rules�dirname�__file__)r r
r
r �inita s z!CheckSuspiciousMarkupBuilder.initc C s | j jS )N)�envZ
found_docs)r r
r
r �get_outdated_docsi s z.CheckSuspiciousMarkupBuilder.get_outdated_docsNc C s dS )N� r
)r r �typr
r
r �get_target_uril s z+CheckSuspiciousMarkupBuilder.get_target_uric C s d S )Nr
)r Zdocnamesr
r
r �prepare_writingo s z,CheckSuspiciousMarkupBuilder.prepare_writingc C s$ d| _ || _t|| �}|j|� d S )NF)� any_issuer �SuspiciousVisitor�walk)r r ZdoctreeZvisitorr
r
r � write_docr s
z&CheckSuspiciousMarkupBuilder.write_docc C sR dd� | j D �}|rN| jdt|�t| j �f � x |D ]}| jjt|�� q6W d S )Nc S s g | ]}|j s|�qS r
)r )�.0�ruler
r
r �
<listcomp>z s z7CheckSuspiciousMarkupBuilder.finish.<locals>.<listcomp>zFound %s/%s unused rules:)�rules�warn�len�logger�info�repr)r Z unused_rulesr/ r
r
r �finishy s
z#CheckSuspiciousMarkupBuilder.finishc C s | j |||�s | j|||� d S )N)�
is_ignored� report_issue)r r
r r r
r
r � check_issue� s z(CheckSuspiciousMarkupBuilder.check_issuec C sd | j }xX| jD ]N}|j |kr q|j|kr*q|j|kr6q|jdk rTt|j| �dkrTqd|_dS W dS )z/Determine whether this issue should be ignored.N� TF)r r1 r r
r �absr )r r
r r r r/ r
r
r r8 � s
z'CheckSuspiciousMarkupBuilder.is_ignoredc C s� | j s| jj� d| _ | j|||� trB| jd| j|||f � n@| jd| jjtj � d�||jtj � d�|j
� jtj � d�f � d| j _ d S )NTz [%s:%d] "%s" found in "%-.120s"�replace� )
r* r4 r5 �write_log_entry�py3r2 r �encode�sys�getdefaultencoding�stripZappZ
statuscode)r �textr r r
r
r r9 � s
z)CheckSuspiciousMarkupBuilder.report_issuec C s� t r>t| jd�}tj|t�}|j| j|||j� g� |j � nJt| jd�}tj|t�}|j| jj
d�||j
d�|j� j
d�g� |j � d S )N�aZabzutf-8) r@ r r �csv�writerr Zwriterowr rD r rA )r r r rE �frH r
r
r r? � s
z,CheckSuspiciousMarkupBuilder.write_log_entryc C s | j jddd� g | _}y tr,t|d�}n
t|d�}W n tk
rL dS X x�ttj|��D ]�\}}t |�dkr�t
d||d |f ��|\}}}} |r�t |�}nd}ts�|j d �}|j d �}| j d �} t
|||| �}
|j|
� q^W |j� | j jd
t | j� � dS ) z�Load database of previously ignored issues.
A csv file, with exactly the same format as suspicious.csv
Fields: document name (normalized), line number, issue, surrounding text
zloading ignore rules... r> )Znonl�r�rbN� z wrong format in %s, line %d: %szutf-8zdone, %d rules loaded)r4 r5 r1 r@ r �IOError� enumeraterG �readerr3 �
ValueError�int�decoder �appendr ) r �filenamer1 rI �i�rowr r r rE r/ r
r
r r � s0
z'CheckSuspiciousMarkupBuilder.load_rules)N)r r r r �namer r4 r# r% r( r) r- r7 r: r8 r9 r? r r
r
r
r r Z s
r c C s& d}x |dkr | r | j } | j}qW |S )z*Obtain line number information for a node.N)�parentr
)�noder r
r
r �
get_lineno� s
rZ c C s: | j dd|�d }| jd|�}|dk r.t| �}| ||� S )a text may be a multiline string; extract
only the line containing the given character index.
>>> extract_line("abc
defgh
i", 6)
>>> 'defgh'
>>> for i in (0, 2, 3, 4, 10):
... print extract_line("abc
defgh
i", i)
abc
abc
abc
defgh
defgh
i
r r r> )�rfind�findr3 )rE �index�p�qr
r
r � extract_line� s
r` c @ s4 e Zd ZdZdd� Zdd� ZeZdd� Zdd � Zd
S ) r+ r c C s t jj| |� || _d S )N)r �GenericNodeVisitorr �builder)r Zdocumentrb r
r
r r � s zSuspiciousVisitor.__init__c C s� t |tjtjf�r�|j� }tt|�p&d| j� | _}t� }xPt |�D ]D}|j
� }t ||j � �}||f|krD| j
j|||� |j||f� qDW d S )Nr )�
isinstancer ZTextZimageZastext�maxrZ �
lastlineno�set�
detect_all�groupr` �startrb r: �add)r rY rE r �seen�matchr r
r
r
r �
default_visit s z SuspiciousVisitor.default_visitc C s
d| _ d S )Nr )re )r rY r
r
r �visit_document s z SuspiciousVisitor.visit_documentc C s
t j�d S )N)r ZSkipNode)r rY r
r
r �
visit_comment s z SuspiciousVisitor.visit_commentN) r r r re r rm Z
unknown_visitrn ro r
r
r
r r+ � s
r+ )r r ) r r �rerG rB Zdocutilsr Zsphinx.buildersr Z sphinx.utilZsphinxZsphinx.util.loggingr � ImportErrorZlogging�compile�UNICODE�VERBOSE�finditerrg � version_infor@ r Zexcelr r rZ r` ra r+ r
r
r
r �<module>* s*