Modifier and Type | Class and Description |
---|---|
protected static class |
CredentialsDescription.BasicCredential
Basic type credentials
|
protected static class |
CredentialsDescription.NTLMCredential
NTLM-style credentials
|
Modifier and Type | Field and Description |
---|---|
protected PageCredentials |
ThrottledFetcher.ThrottledConnection.authentication
Authentication
|
protected PageCredentials |
ThrottledFetcher.ConnectionPoolKey.authentication |
protected PageCredentials |
ThrottledFetcher.ConnectionPool.authentication |
Modifier and Type | Method and Description |
---|---|
PageCredentials |
CredentialsDescription.getPageCredential(String url)
Given a URL, find the right PageCredentials object to use.
|
protected PageCredentials |
WebcrawlerConnector.getPageCredential(String documentIdentifier)
Get the page credentials for a given document identifier (URL)
|
Modifier and Type | Method and Description |
---|---|
protected int |
WebcrawlerConnector.checkFetchAllowed(String documentIdentifier,
String protocol,
String hostIPAddress,
int port,
PageCredentials credential,
IKeystoreManager trustStore,
String hostName,
String[] binNames,
long currentTime,
String pathString,
IVersionActivity versionActivities,
int connectionLimit,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword)
Check robots to see if fetch is allowed.
|
static IThrottledConnection |
ThrottledFetcher.getConnection(IThreadContext threadContext,
String throttleGroupName,
String protocol,
String server,
int port,
PageCredentials authentication,
IKeystoreManager trustStore,
IThrottleSpec throttleDescription,
String[] binNames,
int connectionLimit,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword,
int socketTimeoutMilliseconds,
int connectionTimeoutMilliseconds,
IAbortActivity activities)
Obtain a connection to specified protocol, server, and port.
|
Constructor and Description |
---|
ThrottledFetcher.ConnectionPool(IConnectionThrottler connectionThrottler,
String protocol,
String server,
int port,
PageCredentials authentication,
SSLSocketFactory baseFactory,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword,
int socketTimeoutMilliseconds,
int connectionTimeoutMilliseconds) |
ThrottledFetcher.ConnectionPoolKey(String protocol,
String server,
int port,
PageCredentials authentication,
String trustStoreString,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword,
int socketTimeoutMilliseconds,
int connectionTimeoutMilliseconds) |
ThrottledFetcher.ThrottledConnection(ThrottledFetcher.ConnectionPool myPool,
IFetchThrottler fetchThrottler,
String protocol,
String server,
int port,
PageCredentials authentication,
SSLSocketFactory httpsSocketFactory,
String proxyHost,
int proxyPort,
String proxyAuthDomain,
String proxyAuthUsername,
String proxyAuthPassword,
int socketTimeoutMilliseconds,
int connectionTimeoutMilliseconds)
Constructor.
|