0x1949 Team - FAZEMRX - MANAGER
Edit File: launchpad.cpython-310.pyc
o �v�f�N � @ s� d Z ddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl m Z ddl mZ ejjdkr[ddlmZmZmZ ddlmZ ddlmZmZ eeeeeef d Zndd lmZmZmZmZ dd l mZ ddl mZ d Zzdd l m Z ddl m!Z! e! W n e"y� dZ!Y nw ddl#Z$ddl$Z$ej%�&d�Z'dd� Z(dd� Z)G dd� de$j*j+�Z+da,G dd� de�Z-G dd� de�Z.d!dd �Z/e0d kr�ddl1Z1ddl2Z2dd l3m4Z4 da*da5da6da7G d d � d e1j8�Z9e1�:� dS dS )"z,Crash database implementation for Launchpad.� N)�FailedToDecompressContent)�BytesIO� )� HTTPSHandler�Request� build_opener)�HTTPSConnection)� urlencode�urlopenT)r r r r )r F)� HTTPError)� Launchpadz%~/.cache/apport/launchpad.credentialsc c s` � | D ]*}z|j �� }W n ttfy t�d� Y qw |j}|�d�s*|�d�r-|V qd S )Nz"Broken attachment on bug, ignoring�.txt�.gz)�data�openr r �apport�error�filename�endswith)� attachments� attachment�f�name� r �?/usr/lib/python3/dist-packages/apport/crashdb_impl/launchpad.py�filter_filename, s � ���r c C s t dd� | D ��S )Nc s s$ � | ] }t |j�d ��� �V qdS )�/N)�int� self_link�split�pop)�.0�ir r r � <genexpr>: s �" zid_set.<locals>.<genexpr>)�set)�tasksr r r �id_set8 s r& c @ s e Zd ZdZdd� Zedd� �Zdd� Zedd � �ZdAd d �Z d d� Z dd� Z dd� Z dd� Z dBdd�ZdCdd�Zdd � Zd d � Zd d � Zd!d"� Zd#d$� Zd%d&� Zd'd(� Zd)d*� Zd+d,� Zd-d.� Zd/d0� Zd1d2� Zd3d4� Z dAd5d6�Z d7d8� Z d9d:� Z d;d<� Z d=d>� Z!e"d?d@� �Z#d S )D� CrashDatabasez5Launchpad implementation of crash database interface.c C s t �d�r t �d�|d< |s$|�d�}|r"td |�dd�d }nt}tjj�| ||� |�d�| _ | j r?d|vs>J d ��nd|v sGJ d ��d |v rSd |d | _ nd tj � � | _ || _ || _| jsfJ �d | _d | _t �d|�d��| _| js�tjdd�| _tjtj| jdd� d S d S )a Initialize Launchpad crash database. You need to specify a launchpadlib-style credentials file to access launchpad. If you supply None, it will use default_credentials_path (~/.cache/apport/launchpad.credentials). Recognized options are: - distro: Name of the distribution in Launchpad - project: Name of the project in Launchpad (Note that exactly one of "distro" or "project" must be given.) - launchpad_instance: If set, this uses the given launchpad instance instead of production (optional). This can be overriden or set by $APPORT_LAUNCHPAD_INSTANCE environment. - cache_dir: Path to a permanent cache directory; by default it uses a temporary one. (optional). This can be overridden or set by $APPORT_LAUNCHPAD_CACHE environment. - escalation_subscription: This subscribes the given person or team to a bug once it gets the 10th duplicate. - escalation_tag: This adds the given tag to a bug once it gets more than 10 duplicates. - initial_subscriber: The Launchpad user which gets subscribed to newly filed bugs (default: "apport"). It should be a bot user which the crash-digger instance runs as, as this will get to see all bug details immediately. - triaging_team: The Launchpad user/team which gets subscribed after updating a crash report bug by the retracer (default: "ubuntu-crashes-universe") - architecture: If set, this sets and watches out for needs-*-retrace tags of this architecture. This is useful when being used with apport-retrace and crash-digger to process crash reports of foreign architectures. Defaults to system architecture. �APPORT_LAUNCHPAD_INSTANCE�launchpad_instance�.z://� ����distro�projectz/Must not set both "project" and "distro" optionz0Need to have either "project" or "distro" option� architecturezneed-%s-retraceN�APPORT_LAUNCHPAD_CACHE� cache_dirzlaunchpadlib.cache.)�prefixT)� ignore_errors)�os�getenv�get�default_credentials_pathr r �crashdbr' �__init__r- �arch_tag� packaging�get_system_architecture�options�auth�_CrashDatabase__launchpad�_CrashDatabase__lp_distro�_CrashDatabase__lpcache�tempfile�mkdtemp�atexit�register�shutil�rmtree)�selfr> r= � lp_instancer r r r9 @ s6 !� �zCrashDatabase.__init__c C s | j r| j S tdu r trtj�d� ntj�d� t�d� | j�d�r+| j�d�}nd}t j � | j �}|rAt j � |�sAt �|� ztjd|| jdg| j d d �| _ W | j S ty� } z"t|d �re|j}nt|�}t�d || j � t�d � W Y d}~| j S d}~ww )zReturn Launchpad instance.Nz^ERROR: The python-launchpadlib package is not installed. This functionality is not available. z_ERROR: The python3-launchpadlib package is not installed. This functionality is not available. r+ r) � productionzapport-collect� WRITE_PRIVATEz1.0)�launchpadlib_dir�allow_access_levels�credentials_file�version�contentzZconnecting to Launchpad failed: %s You can reset the credentials by removing the file "%s"�c )r? r �_python2�sys�stderr�write�exitr= r6 r4 �path�dirnamer> �isdir�makedirs� login_withrA � Exception�hasattrrP �strr r )rH r) �auth_dir�e�msgr r r � launchpad� s> �� ��zCrashDatabase.launchpadc c sH � | j st�|D ]}|j�� | j ks t�d| j |j�� �r!|V qd S )Nz ^.+\(%s.*\)$)r- � StopIteration�bug_target_name�lower�re�match)rH r% �tr r r �_get_distro_tasks� s ����z CrashDatabase._get_distro_tasksc C sX | j d u r)| jr| jj| j | _ | j S d| jv r%| jj| jd | _ | j S td��| j S )Nr. z:distro or project needs to be specified in crashdb options)r@ r- rb � distributionsr= �projects� SystemError�rH r r r � lp_distro� s ��zCrashDatabase.lp_distroNc C s> | � |�sJ �| �|�}t||| �� d�}|�� |s J �|S )a� Upload given problem report return a handle for it. This should happen noninteractively. If the implementation supports it, and a function progress_callback is passed, that is called repeatedly with two arguments: the number of bytes already sent, and the total number of bytes to send. This can be used to provide a proper upload progress indication on frontends. )�hostname)�accepts�_generate_upload_blob� upload_blob� get_hostname�close)rH �report�progress_callback� blob_file�ticketr r r �upload� s zCrashDatabase.uploadc C s0 | j �d�}|r|dkrd}|S d}|S d}|S )z/Return the hostname for the Launchpad instance.r) �stagingzstaging.launchpad.netz launchpad.dev� launchpad.net)r= r6 )rH r) ro r r r rs � s ��zCrashDatabase.get_hostnamec C s� i }|� d|�� �}|rt|t�s|�d�}||d< | �� }d|v r'|d }n| j� d�}|sLd|v rAd|| j|d |t|�f S d|| j|t|�f S d|||t|�f S ) a/ Return an URL that should be opened after report has been uploaded and upload() returned handle. Should return None if no URL should be opened (anonymous filing without user comments); in that case this function should do whichever interactive steps it wants to perform.�Title�UTF-8z field.title� SnapSourcer. � SourcePackagez,https://bugs.%s/%s/+source/%s/+filebug/%s?%sz!https://bugs.%s/%s/+filebug/%s?%s) r6 �standard_title� isinstance�bytes�encoders r= r- r )rH ru �handle�args�titlero r. r r r �get_comment_url� s* �� �z CrashDatabase.get_comment_urlc C s dt |� S )z�Return URL for a given report ID. The report is passed in case building the URL needs additional information from it, such as the SourcePackage name. Return None if URL is not available or cannot be determined. z https://bugs.launchpad.net/bugs/)r^ )rH ru �idr r r � get_id_url s zCrashDatabase.get_id_urlc C s� t �� }| jj| }t�d|jtj�}|s!t�d|jtjtjB �}|s'J d��|� d�� d�� dd�� dd �}d |v r`d |v rX|� d d�\}}|� d d �d |� d d�d }n|� d d�d }|� t|�� d |vr�z |j�� |d < W n ty� |j|d < Y nw d|vr�d|jv r�d|d< n'd|jv r�d|d< n d|jv r�d|d< nd|jv r�d|d< n tdt|j� ��d�|j�|d< d|v r�|d |d< |j|d< t|j�D ]z}tj�|j �\} } zd || < W n t y� Y q�w | d k�r|� � || < z || � d�|| < W q� t �y Y q�w | d k�rHz t!j"|d �� � || < W q� t#�yG } zd t| �v�r2� |�$d � |� � || < W Y d!} ~ q�d!} ~ ww t d"|j ��|S )#z>Download the problem report from given ID and return a Report.z(ProblemType:.*)$z^--- \r?$[\r\n]*(.*)z8bug description must contain standard apport format datar+ r} s � s � s s Uname:r �Date� ProblemTypez apport-bug�Bug� apport-crash�Crashzapport-kernelcrash� KernelCrashzapport-package�Packagez(cannot determine ProblemType from tags: � �Tagsr| � OriginalTitle� r r )�fileobjz Not a gzipNzUnknown attachment type: )%r �Reportrb �bugsrf �search� description�S�M�groupr� �replacer �loadr � date_created�ctime�AttributeError�tags� ValueErrorr^ �joinr� r r r4 rW �splitextr r\ �read�decode�UnicodeDecodeError�gzip�GzipFile�IOError�seek) rH r� ru �b�mr� �part1�part2r �key�extr` r r r �download sz � � � � ��zCrashDatabase.downloadFc C s� | j j| }|rt|�� �t|� }nd}t�� } |j| |d� | �� | �d� t r1t � | �} nt � | �} | � � } t r@| �� pC| �� } | �� sJJ �t rP| �� pS| �� } | �� rZJ �| �� dksbJ �|s�|jdd� } | �d� d|v r| | �|d ��� 7 } | |_|�� | j j| }| jdd��d d �}|r�|r�|jd | |_|�� n |s�|j}|j||d � | D ]} |j |p�d | � � d| jdd�| � � dd� q�| � � dS )ah Update the given report ID with all data from report. This creates a text comment with the "short" data (see ProblemReport.write_mime()), and creates attachments for all the bulk/binary data. If change_description is True, and the crash db implementation supports it, the short data will be put into the description instead (like in a new bug). comment will be added to the "short" data. If attachment_comment is given, it will be added to the attachment uploads. If key_filter is a list or set, then only those keys will be added. N)� skip_keysr � text/plain�apport-collectedr� T�r� r} r� z --- �rP �subjectr� F��commentr� � content_typer r �is_patch) rb r� r$ �keysrB � TemporaryFile� write_mime�flushr� rR �email�message_from_file�message_from_binary_file�walk�next�__next__� is_multipart�get_content_typer� �append�_filter_tag_namesr �lp_save� get_payloadr� r� r� � newMessage� addAttachment� get_filenamert )rH r� ru r� �change_description�attachment_comment� key_filter�bugr� �mimera �msg_iter�part�x�textr r r �updatea sR � zCrashDatabase.updater� c C s� t jj�| |||� | jj| }d|v r7|jD ] }|jj� d�r6| j j |d d�|_|� � | jj| } nq|� � r�|jD ]}|jdkrUz|�� W q> tyT Y q>w q>z | �|j�}trd|�� }n|�� }|jdkrtd|_|� � W n ty~ Y nw |�� }|r�t�d|j�}|r�|�d�|kr�|�d �| |�d � |_z|� � W n ty� Y nw | jj| }| � ||� d S ) z�Update the given report ID for retracing results. This updates Stacktrace, ThreadStacktrace, StacktraceTop, and StacktraceSource. You can also supply an additional comment. r � #distribution�r � CoreDump.gz� Undecided�Mediumz,^(.*crashed with SIG.* in )([^( ]+)(\(\).*$)r r+ � N) r r8 r' � update_tracesrb r� � bug_tasks�target�resource_type_linkr rn �getSourcePackager� �has_useful_stacktracer r� � removeFromBugr ri rR r� r� � importancerc �stacktrace_top_functionrf rg r� �_subscribe_triaging_team) rH r� ru r� r� �task�a�fnr� r r r r� � sV � �� � � � zCrashDatabase.update_tracesc C s0 | j j| }t�d|j�}|r|�d�S td��)zNGet 'DistroRelease: <release>' from the given report ID and return it.z"DistroRelease: ([-a-zA-Z0-9.+/ ]+)r+ z)URL does not contain DistroRelease: field)rb r� rf r� r� r� r� )rH r� r� r� r r r �get_distro_release� s z CrashDatabase.get_distro_releasec C s` t �d| j �}| jj| }g }|jD ]}|�|jj�}|s q|j dv r%q|� |� d�� q|S )z5Return list of affected source packages for given ID.z5/%s/(?:(?P<suite>[^/]+)/)?\+source/(?P<source>[^/]+)$)�Invalid� Won't Fix� Fix Released�source) rf �compiler- rb r� r� r� r� r �statusr� r� )rH r� � bug_target_rer� �resultr� rg r r r �get_affected_packages� s � z#CrashDatabase.get_affected_packagesc C s | j j| }|jj| j jjkS )z3Check whether the user is the reporter of given ID.)rb r� �ownerr �me�rH r� r� r r r � is_reporter s zCrashDatabase.is_reporterc C sZ | j j| }|jr dS |jj| j jjkrdS | j jj}|jjD ] }|d |kr* dS q dS )aA Check whether the user is eligible to update a report. A user should add additional information to an existing ID if (s)he is the reporter or subscribed, the bug is open, not a duplicate, etc. The exact policy and checks should be done according to the particular implementation. FT� person_link) rb r� � duplicate_ofr� r r� r � subscriptions�entries)rH r� r� r� �subr r r � can_update s �zCrashDatabase.can_updatec C s^ z| j j| jdd�}t|�W S ty. } zt�dt|�� t� d� W Y d}~dS d}~ww )z}Return an ID set of all crashes which have not been retraced yet and which happened on the current host architecture.� 2011-08-01�r� � created_since�"connecting to Launchpad failed: %srQ N) rn � searchTasksr: r& r\ r r r^ rS rV �rH r� r` r r r �get_unretraced s ��z CrashDatabase.get_unretracedc C s\ z | j jddd�}t|�W S ty- } zt�dt|�� t�d� W Y d}~dS d}~ww )a# Return an ID set of all crashes which have not been checked for being a duplicate. This is mainly useful for crashes of scripting languages such as Python, since they do not need to be retraced. It should not return bugs that are covered by get_unretraced().�need-duplicate-checkr r r rQ N) rn r r& r\ r r r^ rS rV r r r r �get_dup_unchecked* s ��z CrashDatabase.get_dup_uncheckedc C s | j jdd�}t|�S )aO Return an ID set of all crashes which are not yet fixed. The list must not contain bugs which were rejected or duplicate. This function should make sure that the returned list is correct. If there are any errors with connecting to the crash database, it should raise an exception (preferably IOError).r� )r� )rn r r& )rH r� r r r � get_unfixed9 s zCrashDatabase.get_unfixedc C s" | j jjd|| j jd�}|d jS )z�Return the version of given source package in the latest release of given distribution. If 'distro' is None, we will look for a launchpad project . T)� exact_match� source_name� distro_seriesr )rn � main_archive�getPublishedSources�current_series�source_package_version)rH �package�sourcesr r r �_get_source_versionE s � z!CrashDatabase._get_source_versionc sN z�j j| }W n ty Y dS w |jrdS t|j�}�jr�d�j�� � tt� fdd�|��}|sCtt�fdd�|��}|rCdS t |�dkrSt � d�j|� dS |ro|� � }z �� |j�� d �W S tyn Y dS w tt� fd d�|��}|r�tt� fd d�|��}|s�dS dS ttd d� |��}|r�dS ttd d� |��r�dS dS )a Return the package version that fixes a given crash. Return None if the crash is not yet fixed, or an empty string if the crash is fixed, but it cannot be determined by which version. Return 'invalid' if the crash report got invalidated, such as closed a duplicate or rejected. This function should make sure that the returned result is correct. If there are any errors with connecting to the crash database, it should raise an exception (preferably IOError). �invalidz(%s)c s | j dko � | j�� v S �Nr� �r� �bug_target_display_namere �r� ��distro_identifierr r �<lambda>o � z1CrashDatabase.get_fixed_version.<locals>.<lambda>c s | j dko| j�� � j�� kS r )r� rd re r- r rm r r r s s r� r+ zVThere is more than one task fixed in %s %s, using first one to determine fixed versionr c s | j dv o � | j�� v S �N)r� r� �Expiredr r r r r r � r c s | j dvo � | j�� v S r! r r r r r r � r c S � | j dkS r �r� r r r r r � � c S r# )Nr� r$ r r r r r � r% N)rb r� �KeyErrorr� �listr� r- re �filter�lenr �warningr r r r � IndexError) rH r� r� r% � fixed_tasks� fixed_distror� � invalid_tasks�non_invalid_tasksr )r rH r �get_fixed_versionS s\ � �� �� � �z CrashDatabase.get_fixed_versionc C s | j j| j}|r |jS dS )zcReturn master ID for a duplicate bug. If the bug is not a duplicate, return None. N)rb r� r� r� )rH r� r� r r r r� � s zCrashDatabase.duplicate_ofc C s� | j j| }|�rT||ksJ dt|� ��| j j| }|jr2|j}|j}|j|kr2t�d||� dS |jD ]}|jdv rLz|� � W q5 t yK Y q5w q5| j j| }|j d| dd� | j j| }|j rhd|_ |jsn||_|j }t|j�d kr�d | jv r�| jd |vr�| j�d d �|vr�|| jd g |_ |�� d | jv r�| j�d d �|vr�| j j| jd }|j|d� |�� �rd|v s�d|v �r| j||d| g d�d� | j j| }|j dd� } z| �d� W n ty� Y nw z| �d� W n t�y Y nw | |_ z|�� W n t �y Y nw dg} | jjD ]} | jdv�r'�q | � | j � �q t |j �} |j }| � |�} | D ] }|| v �rJ|� |� �q>||_ |�� n|j�r[d|_|j �re|�� dS dS )zlMark a crash id as duplicate of given master ID. If master is None, id gets un-duplicated. z+cannot mark bug %s as a duplicate of itselfzNBug %i was manually marked as a dupe of newer bug %i, not closing as duplicateN)r� zStacktrace.txtzThreadStacktrace.txtz ProcMaps.txtzProcStatus.txtz Registers.txtzDisassembly.txta� Thank you for taking the time to report this crash and helping to make this software better. This particular crash has already been reported and is a duplicate of bug #%i, so is being marked as such. Please look at the other bug report to see if there is any missing information that you can provide, or to see if there is a workaround for the bug. Additionally, any further discussion regarding the bug should occur in the other report. Please continue to report any other bugs you may find.zThis bug is a duplicater� F� �escalation_tag� escalated_tagz invalid �escalation_subscription��personzapport-request-retrace�apport-failed-retracez)Updated stack trace from duplicate bug %i)� Stacktrace�ThreadStacktracer� � Dependencies�ProcMaps� ProcCmdline)r� zbugpattern-needed)zActive DevelopmentzCurrent Stable Release� SupportedzPre-release Freeze)!rb r� r^ r� r� r r* r r� r� r r� �privater� r) � duplicatesr= r6 r� �people� subscriber� r� �remover� rn �seriesr� r� r r$ � difference�_dirty_attributes)rH ru r� � master_idr� �masterr� � master_tags�pr� � tags_to_copyrC � dupe_tags� missing_tags�tagr r r �close_duplicate� s� � �� �� * � �� � � �z CrashDatabase.close_duplicatec C sD | j j| }|jd| dd� | j j| }|jdg |_|�� dS )zpMark a crash id as reintroducing an earlier crash which is already marked as fixed (having ID 'master').z�This crash has the same stack trace characteristics as bug #%i. However, the latter was already fixed in an earlier package version than the one in this report. This might be a regression or because the problem is in a dependent package.z Possible regression detectedr� �regression-retracerN)rb r� r� r� r� )rH r� rG r� r r r �mark_regression s �� z CrashDatabase.mark_regressionc C s` | j j| }| j|jv r.|jdd� }|�| j� ||_z|�� W dS ty- Y dS w dS )zMark crash id as retraced.N)rb r� r: r� rB r� r )rH r� r� r� r r r � mark_retraced# s ��zCrashDatabase.mark_retracedc C s� | j j| }|rUz| �|j�}tr|�� }n|�� }W n ty) |jd }Y nw d|_|� � |j |dd� |j D ]}|j dkrRz|� � W q; tyQ Y q;w q;dS d|jvrg|jdg |_|� � dS dS )z%Mark crash id as 'failed to retrace'.r r� z Crash report cannot be processedr� r� r7 N)rb r� ri r� rR r� r� rc r� r� r� r r� r� r r� )rH r� � invalid_msgr� r� r� r r r �mark_retrace_failed0 s: � �� ��� �z!CrashDatabase.mark_retrace_failedc C s� | j j| }d|v r9|jD ]+}|jj�d�r8| jj|d d�|_z |�� | j j| }W n t y5 Y nw nq d|j v rm|j dd� }|� d� ||_ |�� d|v rm|jD ]}d|jjv rl|j dkrld|_ |�� qX| � ||� dS ) z/Mark crash id as checked for being a duplicate.r r� r� r N� Tracebackr� r� )rb r� r� r� r� r rn r� r� r r� rB r� r� )rH r� ru r� r� r� r r r �_mark_dup_checkedN s8 � �� �z CrashDatabase._mark_dup_checkedc C s� t jj�| |�}|s |S ||d< zt|d �}W n ty# | Y S w |�� }|�d�s/dS |D ] }|�d�rFd|v s@d|v rC dS |S |�� sM |S q1|S ) a� Check if the crash db already knows about the crash signature. Check if the report has a DuplicateSignature, crash_signature(), or StacktraceAddressSignature, and ask the database whether the problem is already known. If so, return an URL where the user can check the status or subscribe (if available), or just return True if the report is known but there is no public URL. In that case the report will not be uploaded (i. e. upload() will not be called). Return None if the report does not have any signature or the crash database does not support checking for duplicates on the client side. The default implementation uses a text file format generated by duplicate_db_publish() at an URL specified by the "dupdb_url" option. Subclasses are free to override this with a custom implementation, such as a real database lookup. � DuplicateOfz/+texts bug:Ts tags:s apport-failed-retraces apport-request-retraceN) r r8 r' �knownr r� �readline� startswith�strip)rH ru �urlr �liner r r rW n s. � ��zCrashDatabase.knownc C sx d|v r|d � � d dkrdS d| jj| j�dd�f }|�t| jj�d��d �d d � |jD �vr:|j |d � dS dS ) z-Subscribe the right triaging team to the bug.� DistroReleaser �UbuntuNz%s~%s� triaging_teamzubuntu-crashes-universer� �~c S s g | ] }t |��d �d �qS )r r, )r^ r )r! r r r r � <listcomp>� s z:CrashDatabase._subscribe_triaging_team.<locals>.<listcomp>r5 ) r rb � _root_urir= r6 r� r^ rZ r rA )rH r� ru r6 r r r r� � s ���z&CrashDatabase._subscribe_triaging_teamc C sn i }d|d � � |d< |�d�}|r|dkr |�d�}|r(|d d| 7 < d|v r;|d d| �|d � 7 < d|v r}|rid |v sMd |v sMd |v rid |d < |�d| j�dd��|d< |d d| 7 < nd|v r}d |d < d|d< |d d7 < d|v r�d|d vr�|d d7 < d|v r�|d |d< g d�}t�� }|j||g d�|d� |�� |�d � |S ) z�Generate a multipart/MIME temporary file for uploading. You have to close the returned file object after you are done with it. z apport-%sr� r� �PackageArchitecture�all� Architecturer� r] �VmCore�CoreDump�LaunchpadPrivate�yes�Private�LaunchpadSubscribe�initial_subscriberr � Subscribersz need-%s-retracerT z need-duplicate-check�DuplicateSignaturer �CheckboxSubmissionzHWDB-Submission) r� r] r� � Regression� Reproducible�TestedUpstream�ProcVersionSignature�Uname�NonfreeKernelModules)r� rh rk )� extra_headersr� �priority_fieldsr ) re r6 r� r= rB r� r� r� r� )rH ru �hdrr� �orderr� r r r rq � sB � � z#CrashDatabase._generate_upload_blobc C s` d}|� � jddd�D ]"}|dv st|�dkr)|dv r)tr"||7 }q |t|�7 }q |d7 }q |S ) zAReplace characters from tags which are not palatable to Launchpadr� �ASCII�ignore)�errorss% abcdefghijklmnopqrstuvwxyz0123456789 r s +-.r* )re r� r) rR �chr)�klassr� �res�chr r r r� � s z CrashDatabase._filter_tag_names�N)FNN)r� )$�__name__� __module__� __qualname__�__doc__r9 �propertyrb ri rn ry rs r� r� r� r� r� r� r� r� r r r r r r0 r� rN rP rQ rS rU rW r� rq � classmethodr� r r r r r' = sJ B ( # U � S4 F q 72r' c @ s e Zd ZdZdd� ZdS )�HTTPSProgressConnectionzWImplement a HTTPSConnection with an optional callback function for upload progress.c C s� t s t�| |� d S d}t|�}d}||k rRt ||� t�� }t�| |||| � � ||7 }t�� }|dkrL|| dk rB|dK }n || dkrL|dL }||k sd S d S )Nr i g �?r+ r )�_https_upload_callbackr �sendr) �time)rH r �sent�total� chunksize�t1�t2r r r r� s$ �z HTTPSProgressConnection.sendN)r� r� r� r� r� r r r r r� s r� c @ s e Zd Zdd� ZdS )�HTTPSProgressHandlerc C s | � t|�S r� )�do_openr� )rH �reqr r r � https_open$ s z HTTPSProgressHandler.https_openN)r� r� r� r� r r r r r� " s r� r{ c C s d}d| }|a tjj�� }tjj�d�}|�dd� |�|� tjj � dd�}|�dd� |� | � � � d �� |�|� t� }tjjd krOtjj|d d �} ntjj|d d �} | �|� t||�� �} | �d d|�� � tt�} | �| �} | � � � d�}|s�J �|S )a Upload blob (file-like object) to Launchpad. progress_callback can be set to a function(sent, total) which is regularly called with the number of bytes already sent and total number of bytes to send. It is called every 0.5 to 2 seconds (dynamically adapted to upload bandwidth). Return None on error, or the ticket number on success. By default this uses the production Launchpad hostname. Set hostname to 'launchpad.dev' or 'staging.launchpad.net' to use another instance for testing. Nzhttps://%s/+storeblob�1zContent-Dispositionz form-data; name="FORM_SUBMIT"� applicationz octet-streamz*form-data; name="field.blob"; filename="x"�asciir F)� mangle_from_z Content-Typez multipart/form-data; boundary=zX-Launchpad-Blob-Token) r� r� r� � multipart� MIMEMultipartr� �MIMEText� add_header�attach�base�MIMEBase� set_payloadr� r� r rS � version_info�major� generator� Generator�BytesGenerator�flattenr �getvalue� get_boundaryr r� r �infor6 ) �blobrv ro rx r[ r �submit� form_blob� data_flat�genr� �openerr� r r r rr ( s. rr �__main__)�patchc @ sT e Zd ZdZdZdd� Zdd� Zedd� �ZdNd d �Z d d � Z dNd d�Z dd� Z dd� Z dd� Zdd� Zdd� Zdd� Zdd � Zd d � Zd d � Zd!d"� Zd#d$� Zd%d&� Zd'd(� Zd)d*� Zd+d,� Zd-d.� Ze � e d/�d0d1� �Z e d2d3� �Z!d4d5� Z"dOd7d8�Z#d9d:� Z$d;d<� Z%d=d>� Z&d?d@� Z'dAdB� Z(dCdD� Z)dEdF� Z*dGdH� Z+dIdJ� Z,e dPdLdM��Z-d6S )Q�_T� coreutilsc C sH t s| �� a t | _ t�� | _| j�� | j�� d| jd< | �d� d S )Nr� r �langpack-o-matic)r8 � _get_instancer r� � ref_report� add_os_info� add_user_info�_create_projectrm r r r �setUpr s z_T.setUpc C s@ | j jj| }|s | j jjj|d |||d |d d� dS dS )z8Create a project using launchpadlib to be used by tests.r� �summaryr� )r� � display_namer r� r� N)r8 rb rk � new_project)rH r r. r r r r� � s ��z_T._create_projectc C s | j �� S )z1Get the Launchpad hostname for the given crashdb.)r8 rs rm r r r ro � s z _T.hostnameFc C s� |st durt S | �� }|�| j� |�� |�� |�� | �|�� d� d|d< d|d< | � | j |�}| � |� | � ||�}| � |dk� t j�d| j|f � |sV|a |S ) z�Generate SEGV crash report. This is only done once, subsequent calls will return the already existing ID, unless force_fresh is True. Return the ID. N�!crash crashed with SIGSEGV in f()u "]¶" �ShortGibberish� a b c d e ÿÿÿ � LongGibberishr z*(Created SEGV report: https://%s/bugs/%i) )� _segv_report�_generate_sigsegv_report�add_package_info� test_packager� � add_gdb_infor� � assertEqualr� �_get_bug_targetr8 � assertTrue� _file_bugrS rT rU ro )rH � force_fresh�r� bug_targetr� r r r �get_segv_report� s$ z_T.get_segv_reportc C s� t durt S t�d�}d|d< d|d< d|d< |�| j� |�� |�� | �|�� d � | � | j |�}| � |� | � ||�}| � |d k� t j�d | j|f � |a |S ) zFGenerate Python crash report. Return the ID. Nr� �/bin/foo�ExecutablePath��Traceback (most recent call last): File "/bin/foo", line 67, in fuzz print(weird) NameError: global name 'weird' is not definedrT zboogus pybogusr� �Hfoo crashed with NameError in fuzz(): global name 'weird' is not definedr z,(Created Python report: https://%s/bugs/%i) )�_python_reportr r� r� r� r� r� r� r� r� r8 r� r� rS rT rU ro )rH r� r� r� r r r �get_python_report� s& � z_T.get_python_reportc C sV |st durt S d}| jjjjd|| jjd�}tj�d| j |j f � |s(|j a |j S )a� File a bug report with an uncommon description. This is only done once, subsequent calls will return the already existing ID, unless force_fresh is True. Example taken from real LP bug 269539. It contains only ProblemType/Architecture/DistroRelease in the description, and has free-form description text after the Apport data. Return the ID. Nzbproblem ProblemType: Package Architecture: amd64 DistroRelease: Ubuntu 8.10 more text and more zmixed description bug)r� r� r� z3(Created uncommon description: https://%s/bugs/%i) ) � _uncommon_description_reportr8 rb r� � createBugrn rS rT rU ro r� )rH r� �descr� r r r � get_uncommon_description_report� s �z"_T.get_uncommon_description_reportc C s� | j �| �� �}| �|d d� | �|d d� | �|d | jd � | �|d | jd � | �|d | jd � | �|�d�| j�d�� | �|�d �| j�d �� t|d �� �}| �|t| j jd t j � � g�� | �|d d � | � |d � d�� | �|d | j� | � |d �| jd �� | �d|d � | �d|d � | �d|d � | �t|d �d� | �d|� | �d|� | �d|� | j �| �� �}t|d �� �}| �|td d d d t j � � g�� d S ) z download()r� r� r| r� r] re rt ru � UserGroupsr� r� �Signal�11r� z/crashr r� r� zf (x=42)r8 � StacktraceTopr9 rg i� r: � Disassembly� Registers�boogus�pybogusr N)r8 r� r� r� r� r6 r$ r r: r r; r< r� r �test_srcpackagerY r� �assertIn� assertGreaterr) r� )rH r� r� r r r �test_1_download� s< �� �z_T.test_1_downloadc C s� | j �| �� �}| �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �|d d� d |d < d |d< d |d< d |d< | j �| �� |d� | j �| �� �}| �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �|d d� | j jj| �� j }| �d|� | �d|� d|d < d |d< d |d< | j �| �� |d� | j �| �� �}| �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �d|� | �|d d� | j jj| �� }d|_ z|� � W n t �y Y nw d|d < | j �| �� |d� | j �| �� �}| �|d d� | j jj| �� }d|_ z|� � W n t �yE Y nw d|d < | j �| �� |d� | j �| �� �}| �|d d� d|d< d|d< d |d < | j �| �� |d � d S ) zupdate_traces()rg r: r� r� r8 r9 r| r� z?? ()r� � long trace�thread even longer trace�bogus�FooBarzI can has a better retrace?r� r� �=read () from /lib/libc.6.so foo (i=1) from /usr/lib/libfoo.so� good retrace!z$crash crashed with SIGSEGV in read()z)crash crashed with SIGSEGV in f() on exitz!good retrace with title amendmentz,crash crashed with SIGSEGV in read() on exitzcrash is crashyz good retrace with custom titler� u "]¶" r� �StacktraceSource�testsN) r8 r� r� r� r� r� � assertNotInrb r� r� r� r� r )rH r� r� r� r r r �test_2_update_traces s� � �z_T.test_2_update_tracesc C s� d}t �d�}||d< | j�|d�}| �|�d�� |�d�|d< | j�|d�}| �|�d�� d|d< | j�|d�}| �|�d �� d S ) z&get_comment_url() for non-ASCII titless 1ä♥2r� r| �* z1/ubuntu/+filebug/42?field.title=1%C3%A4%E2%99%A52r} r� r zC/ubuntu/+source/coreutils/+filebug/42?field.title=1%C3%A4%E2%99%A52N)r r� r8 r� r� r r� )rH r� r� r[ r r r �test_get_comment_urlm s z_T.test_get_comment_urlc C s | j jjdd�}| j jjjd|dd�}|j}| �|dk� tj � d| j |f � t � d�}d �d �|d < d |d < d|d< d|d< d|d< | j j||ddd� | j �|�}| �|d d �d �� | �|d d� | �|d d� | �|d d� | �| j jj| jdg� dS )z"update() with changing description�bashr� z test description for test bug.�testbug�r� r� r� r �(https://%s/bugs/%i) r� s bogus→r} �OneLiner� f() g() h(1)r� �lineone linetwo�ShortGoo�one two three four five six�DpkgTerminalLogs �VarLogDistupgradeBinGoo�NotMeT�r� r� N)r8 rn r� rb r� r� r� r� rS rT rU ro r r� r� r� r� r� r� �rH r� r� r� r� r r r �test_update_description� s0 � �z_T.test_update_descriptionc C s | j jjdd�}| j jjjd|dd�}|j}| �|dk� tj � d| j |f � t � d�}d |d < d |d < d |d< d|d< d|d< | j j||ddd� | j �|�}| �d |� | �d|� | �|d d� | �|d d� | �|d d� | �| j jj| jdg� dS )z update() with appending commentr� r� z Pr0blem --- ProblemType: Bugr� r� r r� r� � bogus→r� r� r� r� r� r� r� � r� �meowFr� r� r� N�r8 rn r� rb r� r� r� r� rS rT rU ro r r� r� r� r� r� r� r� r r r �test_update_comment� s2 � �z_T.test_update_commentc C s | j jjdd�}| j jjjd|dd�}|j}| �|dk� tj � d| j |f � t � d�}d |d < d |d < d |d< d|d< d|d< | j j||ddg d�d� | j �|�}| �d |� | �|d d � | �|d d� | �|d d� | �d|� | �| j jj| jg � dS )zupdate() with a key filterr� r� z test description for test bugr� r� r r� r� r r� r� r� r� r� r� r� r r� r� T)r� r� r� )r� r� r� Nr r� r r r �test_update_filter� s2 � � z_T.test_update_filterc C s" | � | j�| �� �| jd � dS )zget_distro_release()r] N)r� r8 r� r� r� rm r r r �test_get_distro_release� s �z_T.test_get_distro_releasec C s$ | � | j�| �� �| jd g� dS )zget_affected_packages()r N)r� r8 r� r� r� rm r r r �test_get_affected_packages� s �z _T.test_get_affected_packagesc C �, | � | j�| �� �� | �| j�d�� dS )z is_reporter()r+ N)r� r8 r� r� � assertFalserm r r r �test_is_reporter� � z_T.test_is_reporterc C r )z can_update()r+ N)r� r8 r r� r rm r r r �test_can_update� r z_T.test_can_updatec C s� | � | j�| �� �d� | � | j�| �� �d� | �� }| �� }| jdd�}| j�|�}| j�|||� | � | j�|�|� | j�|||� | � | j�|�|� | � | j�|�d� | j�||d� | � | j�|�d� | � | j�|�d� | j�| �� �}| �d|� | �d|� | �d|� | �d|� | �d |� | �d |� | j�t � � ||� | j�|||� | � | j�|�|� | j�t � � |d� | j�t � � |d� | j�||d� | j�t � � |d� | � | j�|�d� | j� ||� | � |� dS ) zduplicate handlingNT�r� r rg r� r; � ProcStatusr� r9 ) r� r8 r� r� r0 r� r� rN r� r r� rP �_verify_marked_regression)rH �segv_id� known_test_id�known_test_id2r� r r r �test_duplicates� sF ��z_T.test_duplicatesc C s� | j �� }| �| �� |� | �| �� |� | j �| �� � | j �� }| �| �� |� | �||�t | �� g��� | �| j � | �� �d� | � | �� � | j �| �� � | j � | �� � | j �� }| �| �� |� | �||�t | �� g��� | �| j � | �� �d� | � | �� � | j �| �� � | j � | �� d� | j �� }| �| �� |� | �||�t | �� g��� | �| j � | �� �d� dS )z-processing status markings for signal crashesNzI don't like your ) r8 r r� r� r� r� rQ r� �unionr$ r0 �_mark_needs_retracerS )rH �unretraced_before�unretraced_afterr r r �test_marking_segv5 s< � � ��z_T.test_marking_segvc C s� | j jjjd| j j| j jdd�}tj�d�pd}t tj�d�d|d��}|jjjd |j|jd d�}|� � }| � |j |� | � |j |� |�|j � |� � }| � |j |� | �||�t|j g��� | �| j �|j �d � d S ) �0processing status markings for a project CrashDB�foozubuntu distro retrace bug�r� r� r� r� r( rz �LP_CREDENTIALSr� �r. r) �barzproject retrace bugN)r8 rb r� r� r: rn r4 �environr6 r' r r� r� r� rQ r� r r$ r0 )rH � distro_bugr) � project_db� project_bugr r r r r �test_marking_projectX s6 � �� �z_T.test_marking_projectc C s� t j�d�pd}tt j�d�d|dd��}|�� }| j�� }| jjjjddg| jj d d �}t d |j � |�� }| j�� }| � ||� | � ||� t|j g��� d S ) r r( rz r �ubuntu�fakearch)r- r) r/ r zneed-fakearch-retracez&ubuntu distro retrace bug for fakearchr z4fake arch bug: https://staging.launchpad.net/bugs/%iN)r4 r r6 r' r r8 rb r� r� rn �printr� r� r r$ )rH r) � fakearch_db�fakearch_unretraced_before� systemarch_unretraced_beforer� �fakearch_unretraced_after�systemarch_unretraced_afterr r r �test_marking_foreign_arch{ s, �� � �z _T.test_marking_foreign_archc C s� | j �� }| �| �� |� | �| �� |� | j �| �� | j� | j �� }| �| �� |� | �||� t | �� g��� | �| j � | �� �d� dS )z2processing status markings for interpreter crashesN) r8 r r� r� r� r� rU r� r� r r$ r0 )rH �unchecked_before�unchecked_afterr r r �test_marking_python� s � z_T.test_marking_pythonc C sp | j dd�}| j�|�}| j�||| � � � d|d< d|d< d|d< | j�||d � | j�|�}| �d |� d S ) z�updating an invalid crash This simulates a race condition where a crash being processed gets invalidated by marking it as a duplicate. Tr r� r� r� r8 r� r9 r� rg N)r� r8 r� rN r� r� )rH r� r� r r r �test_update_traces_invalid� s z _T.test_update_traces_invalidr c G s\ dt j_| �| �� � | j�| �� �}| �|d� | �| �� � | �| j�| �� �d� dS )z�get_fixed_version() for fixed bugs Other cases are already checked in test_marking_segv() (invalid bugs) and test_duplicates (duplicate bugs) for efficiency. z3.14N) r' r � return_value�_mark_report_fixedr� r8 r0 r� �_mark_report_new)rH r� � fixed_verr r r �test_get_fixed_version� s z_T.test_get_fixed_versionc C s( t j�d�pd}tt j�d�d|d��S )zCreate a CrashDB instancer( rz r r$ )r- r) )r4 r r6 r' )r~ r) r r r r� � s ��z_T._get_instancec C s< |j �d�}d|v r|jj|d d�S |r|jj| S | jS )z&Return the bug_target for this report.r. r r� )r= r6 rn r� rb rk )rH �dbru r. r r r r� � s z_T._get_bug_targetNc C sL |du rd}| j �|�}trt�|�}nt�|�}|�� |�� }tr'|�� p*|� � }|� � s1J �tr7|�� p:|� � }|� � rAJ �|� � dksIJ �|d|j dd�� dd� 7 }| j jjj||d �� ||�d |�� �d �} |D ]}|� � rwJ �| jd |�� d|j dd�|�� d d� qo|d �� D ]} | j jj| } | r�| j| d� q�| jS )zLFile a bug report for a report. Return the bug ID. Nzsome descriptionr� z Tr� r} r� r� r| r r� Fr� rm r5 )r8 rq rR r� r� r� rt r� r� r� r� r� r� r� rb r� r� r r6 r� r� r� r@ rA r� ) rH r� ru r� r� ra r� �headerr� r� � subscriberr r r r r� � sD � � �z _T._file_bugc C s> | j jj| }| j j|jvr |j| j jg |_|�� dS dS )z$Mark a report ID as needing retrace.N)r8 rb r� r: r� r� r� r r r r s �z_T._mark_needs_retracec C s6 | j jj| }d|jvr|jdg |_|�� dS dS )z,Mark a report ID as needing duplicate check.r N)r8 rb r� r� r� r� r r r �_mark_needs_dupcheck$ s �z_T._mark_needs_dupcheckc C �B | j jj| }t|j�}t|�dksJ �|d }d|_|�� dS )z Close a report ID as "fixed".r+ r r� N�r8 rb r� r' r� r) r� r� �rH r� r� r% rh r r r r2 , � z_T._mark_report_fixedc C r: )z Reopen a report ID as "new".r+ r �NewNr; r< r r r r3 6 r= z_T._mark_report_newc C s | j jj| }| �d|j� dS )z.Verify that report ID is marked as regression.rO N)r8 rb r� r� r� r� r r r r @ s z _T._verify_marked_regressionc C sv t j�d�pd}tt j�d�d|d��}| �|jd� t�d�}d|d < d |d < |�� |� � | �|� � d � | � ||�}| �|j d� | � ||�}| �|d k� tj�d| j|f � |�|�}d|d< d|d< d|d< |�||d� |�|�}| �|�|�d� |�||| �� � | �|�|�| �� � | �|�|�d� |�||d� | �|�|�d� | �|�|�d� dS )z7reporting crashes against a project instead of a distror( rz r r� r Nr� r� r� r� rT r� r r� r� r� r� r8 r� r9 r� r )r4 r r6 r' r� r- r r� r� r� r� r� r r� r� rS rT rU ro r� r� r0 rN r� r� )rH r) r8 r� r� r� r r r � test_projectF sB �� � z_T.test_projectc C sH | j �| �� �}| �|d d� | �|d d� | �|d �d�� dS )z*download() of uncommon description formatsr� r� re �amd64r] zUbuntu N)r8 r� r� r� r� rY )rH r� r r r �test_download_robustnessv s z_T.test_download_robustnessc sx t j�d�pd}tt j�d�d|ddd��}d}|jj|jd j� d }zst||d �D ]N}|d 7 }t j � d | � |� t �� || �� � |jj| �� }|jd |jv }t� fdd�|jD ��}|dkrp| �|� | �|� q,| �|� | �|� q,W t||| �D ]}t j � d| � |� t �� |d� q�n t||| �D ]}t j � d| � |� t �� |d� q�w t j � d� dS )z,Escalating bugs with more than 10 duplicatesr( rz r r$ � omgkittenszapport-hackers)r- r) r2 r4 r r4 iV' � r+ z%i r2 c s g | ]}|j � k�qS r )r� )r! �s�rI r r ra � s z&_T.test_escalation.<locals>.<listcomp>r1 zR%i N� )r4 r r6 r' rb r@ r= r �rangerS rT rU rN r r� r� r� r� �anyr r r� )rH r) r6 �count� first_dupr� �has_escalation_tag�has_escalation_subscriptionr rE r �test_escalation sB �� ���z_T.test_escalationc C s� | � | �� � | j�� }| �| �� |� | jjj| �� }|jr'd|_|�� |j d }| jjj d |_ |�� |j | jjj d d� | j�| �� �}| j�| �� |� | j�� }| �| �� |� | �||�t| �� g��� | jjj| �� }| �|j d jd� | �|j d jd� | �|j d jd� | �|j d jd � | �|j d jd� | �|j d jd � | �| j�| �� �d � d S ) z9source package task fixup for marking interpreter crashesFr r$ r� )r� r> r� r+ zcoreutils (Ubuntu)r� N)r9 r� r8 r r� rb r� r> r� r� rj r� �addTaskrk r� rU r� r� r r$ rd r� r� r0 )rH r- r� rh r� r. r r r � test_marking_python_task_mangle� s4 � z"_T.test_marking_python_task_mangler� c C s0 d}t �� }tj�� }z�t�� }t�t j |� t � |� t dd�� }|� d� W d � n1 s1w Y t�g d��dksAJ �t j�d�sIJ �tjg d�tjd � t j�d �s[J �t�d g� tjg d �tjd �dksoJ �t j�|d�|d < t j�|d �f|d< ||d< |�� W t � |� |S t � |� w )a Create a test executable which will die with a SIGSEGV, generate a core dump for it, create a problem report with those two arguments (ExecutablePath and CoreDump) and call add_gdb_info(). Return the apport.report.Report. N�crash.c�wzS int f(x) { int* p = 0; *p = x; return x+1; } int main() { return f(42); } )�gccz-grP z-o�crashr rS )�gdbz--batch�--ex�runrU zgenerate-core-file corez./crash)�stdout�core�sync)�readelfz-nrX r� rg r� )r4 �getcwdr ru r� rB rC rD rE rF rG �chdirr rU � subprocess�callrW �exists�PIPE� check_callr� r� )r~ �signal�workdir�orig_cwd�pr�fdr r r r� � s: � � �� �z_T._generate_sigsegv_report)Fr� )r� ).r� r� r� r� r� r� r� r� ro r� r� r� r� r� r� r� r r r r r r r r r# r, r/ r0 r� �objectr' r5 r� r� r� r� r r9 r2 r3 r r? rA rM rO r� r r r r r� h sV # $"Q #!6## : 0 ")r� )Nr{ );r� rB rD �os.pathr4 rf r� rS r� r� rF �httplib2r �ior r� r� �urllib2r r r �httplibr �urllibr r rR �urllib.request� urllib.parse� http.client�launchpadlib.errorsr �launchpadlib.launchpadr � ImportError�apport.crashdbr rW � expanduserr7 r r& r8 r' r� r� r� rr r� �unittestr] � unittest.mockr� r� r� r� �TestCaser� �mainr r r r �<module> s| H � F 7 � � � � � � ��