0x1949 Team - FAZEMRX - MANAGER
Edit File: fetch.cpython-310.pyc
o ���c� � @ s� d dl Z d dlZd dlZd dlmZ d dlmZ d dlmZ d dl m Z m Z G dd� de �Z G dd � d e �ZG d d � d e �Zd d i ddddd dddf dd�Zdd� Zd dd�Zd dd�Zd dd�Zdd � Zed krteejd d� � dS dS )!� N)� OptionParser)� DeferredList)� deferToThread)� iteritems� networkStringc @ s e Zd ZdS )� FetchErrorN)�__name__� __module__� __qualname__� r r �5/usr/lib/python3/dist-packages/landscape/lib/fetch.pyr s r c @ s$ e Zd Zdd� Zdd� Zdd� ZdS )� HTTPCodeErrorc C � || _ || _d S �N)� http_code�body)�selfr r r r r �__init__ � zHTTPCodeError.__init__c C � d| j S )Nz Server returned HTTP code %d�r �r r r r �__str__ � zHTTPCodeError.__str__c C r )Nz <HTTPCodeError http_code=%d>r r r r r �__repr__ r zHTTPCodeError.__repr__N)r r r r r r r r r r r s r c @ s0 e Zd Zdd� Zdd� Zdd� Zedd� �Zd S ) � PyCurlErrorc C r r )� error_code�_message)r r �messager r r r r zPyCurlError.__init__c C � d| j | jf S )Nz Error %d: %s�r r r r r r r # s zPyCurlError.__str__c C r )Nz <PyCurlError args=(%d, '%s')>r r r r r r & s �zPyCurlError.__repr__c C s | j S r )r r r r r r * s zPyCurlError.messageN)r r r r r r �propertyr r r r r r s r F� � iX Tc C s ddl } t|t�s|�d�}t�|�} t�� }|du r | �� }|�| jt t | ��� |rF|�| j d� |rF|�| j t |�� |�| j| j� |rV| �d�rV|�| jt |�� |rh|�| jdd� tt|��D �� |rq|�| jd� | rz|�| jd � | dur�|�| jt | �� | dur�|�| jt | �� |�| jd � |�| j|� |�| jd � |�| j |� |�| j d � |�| j |j � |�| j d� |�| j!d � z|�"� W n | j#y� } z t$|j%d |j%d ��d}~ww |�&� }|�'| j(�}|d kr�t)||��|S ) aC Retrieve a URL and return the content. @param url: The url to be fetched. @param post: If true, the POST method will be used (defaults to GET). @param data: Data to be sent to the server as the POST content. @param headers: Dictionary of header => value entries to be used on the request. @param curl: A pycurl.Curl instance to use. If not provided, one will be created. @param cainfo: Path to the file with CA certificates. @param insecure: If true, perform curl using insecure option which will not attempt to verify authenticity of the peer's certificate. (Used during autodiscovery) @param follow: If True, follow HTTP redirects (default True). @param user_agent: The user-agent to set in the request. @param proxy: The proxy url to use for the request. r Nzutf-8Tzhttps:c S s g | ]}d | �qS )z%s: %sr )�.0�pairr r r � <listcomp>\ s zfetch.<locals>.<listcomp>F� � s gzip,deflate�� )*�pycurl� isinstance�bytes�encode�io�BytesIO�Curl�setopt�URLr �str�POST� POSTFIELDSIZE�len� READFUNCTION�read� startswith�CAINFO� HTTPHEADER�sortedr �SSL_VERIFYPEER�FOLLOWLOCATION� USERAGENT�PROXY� MAXREDIRS�CONNECTTIMEOUT�LOW_SPEED_LIMIT�LOW_SPEED_TIME�NOSIGNAL� WRITEFUNCTION�write�DNS_CACHE_TIMEOUT�ENCODING�perform�errorr �args�getvalue�getinfo� HTTP_CODEr )�url�post�data�headers�cainfo�curl�connect_timeout� total_timeout�insecure�follow� user_agent�proxyr* �output�input�er r r r r �fetch/ s\ � �� r_ c O s t tg| �R i |��S )z]Retrieve a URL asynchronously. @return: A C{Deferred} resulting in the URL content. )r r_ )rL �kwargsr r r � fetch_async� s ra c K sV g }| D ] }t |fi |��}|r|�||� |r |�||� |�|� qt|ddd�S )a9 Retrieve a list of URLs asynchronously. @param callback: Optionally, a function that will be fired one time for each successful URL, and will be passed its content and the URL itself. @param errback: Optionally, a function that will be fired one time for each failing URL, and will be passed the failure and the URL itself. @return: A C{DeferredList} whose callback chain will be fired as soon as all downloads have terminated. If an error occurs, the errback chain of the C{DeferredList} will be fired immediatly. T)�fireOnOneErrback� consumeErrors)ra � addCallback� addErrback�appendr )�urls�callback�errbackr` �resultsrP �resultr r r �fetch_many_async� s rl c C s. | � d��d�d }|durtj�||�}|S )z�Return the last component of the given C{url}. @param url: The URL to get the filename from. @param directory: Optionally a path to prepend to the returned filename. @note: Any trailing slash in the C{url} will be removed �/���N)�rstrip�split�os�path�join)rP � directory�filenamer r r �url_to_filename� s rv c s. � fdd�}�fdd�}t | f||d�|��S )aJ Retrieve a list of URLs and save their content as files in a directory. @param urls: The list URLs to fetch. @param directory: The directory to save the files to, the name of the file will equal the last fragment of the URL. @param logger: Optional function to be used to log errors for failed URLs. c s, t |� d�}t|d�}|�| � |�� d S )N�rt �wb)rv �openrG �close)rR rP ru �fdrw r r rG � s z fetch_to_files.<locals>.writec s � r � d|t | j�f � | S )Nz Couldn't fetch file from %s (%s))r3 �value)�failurerP )�loggerr r � log_error� s �z!fetch_to_files.<locals>.log_error)rh ri )rl )rg rt r~ r` rG r r )rt r~ r �fetch_to_files� s r� c C sZ t � }|jddd� |jddd� |�d� |�| �\}\}tt||j|j|jd�� d S ) Nz--post� store_true)�actionz--datar" )�defaultz--cainfo)rQ rR rT )r � add_option� parse_args�printr_ rQ rR rT )rL �parser�optionsrP r r r �test� s �r� �__main__r' )NNr )rq �sysr. �optparser �twisted.internet.deferr �twisted.internet.threadsr �twisted.python.compatr r � Exceptionr r r r_ ra rl rv r� r� r �argvr r r r �<module> s, �R �