Get Gentoo!
gentoo.org sites
gentoo.org Wiki Bugs Forums Packages
Planet Archives Sources
Infra Status
summaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
...
* Fix: segget freezes while trying to download files via proxy-fetcherKostyantyn Ovechko2010-07-2019-117/+342
|
* Improve network selection algorithm.Kostyantyn Ovechko2010-07-188-82/+271
| | | | | | | | | 0) Segget starts from the highest priority networks. 1) Segget tries local mirrors and mirrors provided by proxy-fetchers first. 2) If no success and requests to proxy-fetchers are allowed => segget selects proxy-fetcher and requests from it. 3) If proxy-fetcher replied that file has been downloaded => segget starts from step 1 again. 4) If no download started till now and remote-mirrors are allowed => segget selects a remote network and selects a mirror from it. 5) If no success on step 4, segget switches to lower priority networks
* Split provide_segment function into 3 partsKostyantyn Ovechko2010-07-184-120/+180
|
* Add responses from proxy-fetcther to its clientsKostyantyn Ovechko2010-07-187-62/+131
| | | | | | | | | R_PF_BE_MORE_PATIENT R_PF_ERROR_ADDING_TO_PROXY_QUEUE R_PF_ADDED_TO_PROXY_QUEUE R_PF_ALREADY_WAS_IN_QUEUE R_PF_DOWNLOADED R_PF_FAILED
* Implement check if distfile is already in the proxy-fetcher's queue. If so ↵Kostyantyn Ovechko2010-07-176-14/+67
| | | | don't add it.
* Add options from tuiclient.conf fileKostyantyn Ovechko2010-07-1725-68/+655
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [ui_server] tuiclient monitors segget's activity by establishing tcp connection with segget daemon (ui_server part of it). UI_IP Define an ip address segget uses to provide access for tuiclients. The parameter should be a string holding your host dotted IP address. Default: ui_ip=127.0.0.1 UI_PORT Define a port segget uses to provide access for tuiclients. The parameter should be an integer. Minimum value: 1 Maximum value: 65535 Default: ui_port=9999 [logs] LOGS_DIR Define a dir to store log files. Default: logs_dir=./logs GENERAL_LOG_FILE Define a file name to store general log. Default: general_log_file=tuiclient.log ERROR_LOG_FILE Define a file name to store error log. Default: error_log_file=segget.log DEBUG_LOG_FILE Define a file name to store debug log. Default: debug_log_file=segget.log
* Add section [network_proxy_fetcher]Kostyantyn Ovechko2010-07-1712-77/+163
| | | | | | | | | | | | | | PROXY_FETCHER_IP This option is active only when NETWORK_MODE=1, in other cases it's ignored. Specify IP addres of the proxy-fetcher Default: proxy_fetcher_ip=none PROXY_FETCHER_PORT This option is active only when NETWORK_MODE=1, in other cases it's ignored. Specify proxy-fetcher address Default: proxy_fetcher_port=3131
* Fix: warnings with unused variablesKostyantyn Ovechko2010-07-176-7/+14
|
* Fix: string with .o files in MakefileKostyantyn Ovechko2010-07-161-1/+2
|
* Fix: Generate one .o per .cpp, and link them together.Kostyantyn Ovechko2010-07-1626-50/+146
|
* Add to segget.conf file section [provide_proxy_fetcher_to_others].Kostyantyn Ovechko2010-07-1418-125/+484
| | | | | | | | | | | | | | | | | | | | | NOTE: Proxy-fetcher downloads distfiles requested by other segget daemons. File will be ignored if it's already downloaded. TO-DO: Proxy-fetcher needs to check if distfile is already in the download list. [provide_proxy_fetcher_to_others] PROVIDE_PROXY_FETCHER_IP Define an ip address segget will use to provide access for tuiclients. The parameter should be a string holding your host dotted IP address. Default: provide_proxy_fetcher_ip=127.0.0.1 PROVIDE_PROXY_FETCHER_PORT Define a port segget will use to provide access for tuiclients. The parameter should be an integer. Minimum value: 1 Maximum value: 65535 Default: provide_proxy_fetcher_port=9777
* Add section [ui_server] to segget.conf fileKostyantyn Ovechko2010-07-142-78/+0
| | | | | | | | | | | | | | | | | | | [ui_server] tuiclient monitors segget's activity by establishing tcp connection with segget daemon (ui_server part of it). UI_IP Define an ip address segget will use to provide access for tuiclients. The parameter should be a string holding your host dotted IP address. Default: ui_ip=127.0.0.1 Define a port segget will use to provide access for tuiclients. The parameter should be an integer. Minimum value: 1 Maximum value: 65535 Default: ui_port=9999
* Add help window to tuiclientKostyantyn Ovechko2010-07-1314-274/+725
|
* Add scrolling to tuiclientKostyantyn Ovechko2010-07-127-145/+202
|
* Add colors to tuiclient interfaceKostyantyn Ovechko2010-07-121-7/+90
|
* Separate text based ui from segget into a tuiclient.Kostyantyn Ovechko2010-07-1112-27/+617
| | | | | Prepare segget for daemon phase. Segget gets ui_server interface and tuiclient in curses as a separate app.
* Move show_progress() to a threadKostyantyn Ovechko2010-07-1019-164/+779
|
* Failprove .cpp files and log errors to error.logKostyantyn Ovechko2010-07-0523-450/+661
|
* Failprove networkbroker.cpp and log errors to error.logKostyantyn Ovechko2010-07-044-13/+29
|
* Add option [provide_mirror_to_others].provide_mirror_files_restrict_list_on ↵Kostyantyn Ovechko2010-07-045-6/+71
| | | | | | | | | | | | to segget.conf file SYNOPSIS: PROVIDE_MIRROR_FILES_RESTRICT_LIST_ON= 0 | 1 If PROVIDE_MIRROR_DIR=none this option will be ignored. - If set to 1, segget will compare distfile name with the list of forbiden patterns from the restricted.conf file. If distfile name contains any of the patterns, no symlink will be provided to this distfile. Default: provide_mirror_files_restrict_list_on=0
* Add option [provide_mirror_to_others].provide_mirror_dirKostyantyn Ovechko2010-07-049-29/+85
| | | | | | | | [provide_mirror_to_others] PROVIDE_MIRROR_DIR Define a dir to store distfiles for mirroring. Default: provide_mirror_dir=./provide_mirror_dir
* Merge 2 vectors: network_array[network_num].mirror_list and ↵Kostyantyn Ovechko2010-07-045-14/+5
| | | | network_array[network_num].benchmarked_mirror_list
* Add option [network_mirrors].ONLY_LOCAL_WHEN_POSSIBLE to network#.conf files.Kostyantyn Ovechko2010-07-0417-258/+194
| | | | | | | | | | | | SYNOPSIS: ONLY_LOCAL_WHEN_POSSIBLE=0 | 1 If NETWORK_USES_OWN_MIRROR_LIST_ONLY_ON=0 this option will be ignored. - If set to 1, segget will not use remote mirrors with equal or lower priority until all mirrors in network0_mirrors.conf file have failed. - If set to 0, segget will use remote mirrors with equal priority or mirrors with lower priority when this network has NO free connections (see option NETWORK_MAX_CONNECTIONS in [network_connections] section of this file). Default: only_local_when_possible=1
* Add support for local mirrorsKostyantyn Ovechko2010-07-0422-125/+443
| | | | | | | | | | | network#.conf :: [network_mirrors] SYNOPSIS: NETWORK_USES_OWN_MIRROR_LIST_ONLY_ON=0 | 1 - If set to 1, segget will replace mirror list provided by portage system with the list from network1_mirrors.conf - If set to 0, segget will use ONLY mirror list provided by portage system, and will NOT use the list from network1_mirrors.conf file Default: use_own_mirror_list_only_on=0
* Add competitions among networks with equal priority levels.Kostyantyn Ovechko2010-07-033-6/+25
| | | | Network with the smallest active_connections_num wins.
* Add support for several networks to seggetKostyantyn Ovechko2010-07-0328-333/+1026
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add [networks] section to segget.conf [networks] network0_priority=10 network1_priority=0 network2_priority=0 network3_priority=0 network4_priority=0 network5_priority=0 network6_priority=0 network7_priority=0 network8_priority=0 network9_priority=0 2) Following options moved from segget.conf to network#.conf [network_bind] bind_interface=none [network_connections] max_connections=2 connection_timeout=15 ftp_response_timeout=180 timeout=500 low_connection_speed_limit=1000 low_connection_speed_time=10 max_connection_speed=3000 [network_protocols] [network_user_data] user_agent=segget [network_proxy] proxy_ip_or_name=none proxy_port=3128 proxy_user=none proxy_password=none proxy_off=1 [network_mirrors] use_own_mirror_list_on=0
* Add option [pkg_list].del_pkg_list_when_dld_finishedKostyantyn Ovechko2010-06-307-19/+56
| | | | | | | | SYNOPSIS: del_pkg_list_when_dld_finished=0 | 1 - If del_pkg_list_when_dld_finished set to 1: Segget deletes pkg.list file, after all distfiles were successfuly fetched. Default: del_pkg_list_when_dld_finished=1
* Add option pkg_list_dirKostyantyn Ovechko2010-06-294-2/+10
| | | | | | | PKG_LIST_DIR Define a dir with pkg.list file Default: pkg_list_dir=./
* Add avg speed measurement for connections.Kostyantyn Ovechko2010-06-2811-133/+313
| | | | connection->avg_speed=connection->total_dld_bytes/time_left_from(connection->start_time);
* Add option [connections].current_speed_time_interval_msecs to segget.confKostyantyn Ovechko2010-06-2819-19/+2074
| | | | | | | | | | | CURRENT_SPEED_TIME_INTERVAL_MSECS segget transfers may have bursty nature of their traffic. Therefore, while measuring current speed, segget actually calculates average speed during current_speed_time_interval_msecs time interval, defined in milliseconds. Min limit:100 Max limit: 60000 Default: current_speed_time_interval_msecs=1000
* Make time measurement more precise - in millisecondsKostyantyn Ovechko2010-06-282-11/+23
|
* Failprove segget.cpp and log errors to error.logKostyantyn Ovechko2010-06-272-151/+210
|
* Failprove mirror.cpp and log errors to error.logKostyantyn Ovechko2010-06-271-33/+55
|
* Failprove distfile.cpp and log errors to error.logKostyantyn Ovechko2010-06-272-161/+212
|
* Failprove checksum.cpp and log errors to error.logKostyantyn Ovechko2010-06-271-69/+104
|
* Failprove pkg.cpp and log errors to error.logKostyantyn Ovechko2010-06-271-12/+24
|
* Failprove segment.cpp and log errors to error.logKostyantyn Ovechko2010-06-275-110/+133
|
* Failprove settings.cpp and log errors to error.log fileKostyantyn Ovechko2010-06-271-35/+41
|
* Failprove stats.cpp and log errors to error.logKostyantyn Ovechko2010-06-271-8/+14
|
* Failprove tui.cpp and log errors to error.logKostyantyn Ovechko2010-06-272-47/+119
|
* Failprove config.cpp.Kostyantyn Ovechko2010-06-274-62/+156
| | | | | | | | | 1) Catch exception if segget.conf can't be opened. 2) Catch exceptions if while reading segget.conf. 3) Check if settings suplied by segget.conf can be converted to appropriate type. 4) Check if settings fit allowed range from min_limit to max_limit. 5) In case there was an error apply default settings. 6) Log all described errors.
* Add explanation for options [mirrors].use_benchmark_stats and ↵Kostyantyn Ovechko2010-06-273-9/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [mirrors].benchmark_oblivion SYNOPSIS: use_benchmark_stats=0 | 1 If use_benchmark_stats=1 statistics on mirrors is used to rate them and therefore improve performance. Each time connection from a particular mirror closes mirror->dld_time, and mirror->dld_size get increased (in case of unsuccessful connection only time gets increased), so avg speed for a mirror can be calculated: mirror->avg_speed=mirror->dld_size/mirror->dld_time. (1) When new segment is going to be started segget goes through the list of the mirrors distfile/segment has, and asks each mirror for self_rating: "ulong self_rating=mirror->mirror_on_the_wall();". This way segget chooses a mirror with the best self_rating. To calculate self_rating mirrors use the following formula: self_rating=dld_time/dld_size*honesty. (2) So mirrors actually say how bad they are. Even mirrors can have critical times, so to give mirrors another chance honesty was added to the formula (2). honesty can get values in interval (0,1]. Each time connection from a mirror opens or closes mirror sets its honesty=1. If mirror was asked for self_evaluation with mirror->mirror_on_the_wall(), but wasn't chosen its honesty decreases somewhat (see [mirror].benchmark_oblivion option), so next time it will lie little bit more about how bad it's. Default: use_benchmark_results=1 ***Note: at the moment use_benchmark_results can NOT be set to 0 BENCHMARK_OBLIVION benchmark_oblivion option allows to adjust how fast segget "forgets" benchmarking statistics on mirrors performance. Each time mirror->mirror_on_the_wall() called, mirror decreases its honesty (to have more chances next time) using the following formula: honesty=honesty*100/(100+settings.benchmark_oblivion) (3) Therefore, setting benchmark_oblivion=100 will make mirror look twice less bad next time mirror->mirror_on_the_wall() called. Default: benchmark_oblivion=5
* Change default values for settings.Kostyantyn Ovechko2010-06-244-32/+32
| | | | | | | | max_tries=10 ftp_response_timeout=180 timeout=500 max_connections_num_per_mirror=1 proxy_port=3128.
* Add [logs] section to segget.confKostyantyn Ovechko2010-06-249-96/+155
| | | | | | | | | | | | | | | | | | | | | | LOGS_DIR Define a dir to store log files. Default: logs_dir=./logs GENERAL_LOG_FILE Define a file name to store general log. Default: general_log_file=segget.log ERROR_LOG_FILE Define a file name to store error log. Default: error_log_file=segget.log DEBUG_LOG_FILE Define a file name to store debug log. Default: debug_log_file=debug.log
* Add [user-data].user_agent option to segget.confKostyantyn Ovechko2010-06-233-2/+14
| | | | | | | | USER_AGENT Set the User-Agent: header in the http request sent to the remote server. This can be used to fool servers or scripts. Default: user_agent=segget
* Add [proxy] section settings to segget.confKostyantyn Ovechko2010-06-234-17/+106
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | PROXY_IP_OR_NAME Specify a proxy to use (address and port). Set HTTP proxy to use. The parameter should be a string holding the proxy host name or dotted IP address. To specify port number in this string, append :[port] to the end of the host name. The proxy string may be prefixed with [protocol]:// since any such prefix will be ignored. The proxy's port number may optionally be specified with the separate option. If not specified, by default port 1080 will be used for proxies. When you tell segget to use an HTTP proxy, segget will transparently convert operations to HTTP even if you specify an FTP URL etc. Segget respects the environment variables http_proxy, ftp_proxy, all_proxy etc, if any of those are set. The PROXY option does however override any possibly set environment variables. Default: proxy_ip_or_name=none PROXY_PORT Set the proxy port to connect to unless it is specified in the PROXY option. Default: proxy_port=1080 PROXY_USER Set user name to use for the transfer while connecting to Proxy. The PROXY_USER option should be used in same way as the PROXY_PASSWORD is used. In order to specify the password to be used in conjunction with the user name use the PROXY_PASSWORD option. Default: proxy_user=none PROXY_PASSWORD Set password to use for the transfer while connecting to Proxy. The PROXY_PASSWORD option should be used in conjunction with the PROXY_USER option. Default: proxy_password=none SYNOPSIS: proxy_off=0 | 1 Setting the proxy_off=1 will explicitly disable the use of a proxy, even if there is an environment variable set for it. Default: proxy_off=1
* Add [mirrors].benchmark_oblivion option to segget.confKostyantyn Ovechko2010-06-233-29/+78
| | | | | | | | | benchmark_oblivion option allows to adjust how fast segget "forgets" benchmarking statistics on mirrors performance. Default: benchmark_oblivion=5 Note: settings.forgetness option replaced by settings.benchmark_oblivion.
* Add Tmirror class for mirror benchmarking and option ↵Kostyantyn Ovechko2010-06-237-14/+152
| | | | | | | | [mirrors].max_connections_num_per_mirror Segget gathers stats on mirrors and uses it to rate them and therefore improve performance. settings.forgetness option (by default set to 5) allows to adjust how fast segget "forgets" about bad performance of a mirror.
* Add option [connections].bind_interfaceKostyantyn Ovechko2010-06-223-3/+31
| | | | | | | | | BIND INTERFACE / IP Pass a string as parameter. This sets the interface name to use as outgoing network interface. The name can be an interface name, an IP address, or a host name. No binding is set by default. default: bind_interface=none
* Add [connections].low_connection_speed_limit, ↵Kostyantyn Ovechko2010-06-224-69/+112
| | | | | | | | | | | | | | | | | [connections].low_connection_speed_time LOW_CONNECTION_SPEED_LIMIT Define the low speed limit for connection. Pass a long as parameter. It contains the transfer speed in bytes per second that the transfer should be below during LOW_CONNECTION_SPEED_TIME seconds to consider it too slow and abort. default: low_connection_speed_limit=1000 LOW_CONNECTION_SPEED_TIME Pass a long as parameter. It contains the time in seconds that the transfer should be below the LOW_SPEED_LIMIT to consider it too slow and abort. default: low_connection_speed_time=10