• 13
  • Microsoft Proxy Server Directly and Via SNMP
    • Microsoft Proxy Server Logs
      • Text File Logs
        • Data Field Definitions
        • Verbose Fields and Regular Fields
        • Log Cautions
      • Database Logging
        • Preparing the Database
        • DSN
    • Other Log Files
    • SNMP
    • Summary

13

Microsoft Proxy Server Directly and Via SNMP

There are two primary methods to monitor the activity of Microsoft Proxy Server. The first and more common is activity logging, which can be done through standard comma delimited text files, or to a qualified database through ODBC drivers (Open DataBase Connectivity). The second method, and one which will be available to a small minority of network administrators, is monitoring through SNMP (Simple Network Management Protocol). SNMP is a protocol service that can be installed in Windows NT. The purpose of SNMP is to serve out monitoring and sometimes controlling data to a remote workstation(s) so that services running on an NT server can be monitored and controlled from an outside location.

Utilizing SNMP requires client software on the workstation monitoring the SNMP data stream. Several companies make SNMP software, but it's normally pretty expensive stuff. This chapter doesn't delve too deeply into the realm of SNMP. It's primary focus is to explain the structure of the Microsoft Proxy Server logs and also explain the principles of how to connect Microsoft Proxy Server to a database server engine so that log information can be saved to a database file via ODBC.

Microsoft Proxy Server Logs

Microsoft Proxy Server can log information to a text file or to a database through ODBC drivers to engines, such as an Access Server or an SQL Server. Logging to a text file is a very simple process and will give a network administrator a quick way of seeing what transactions have transpired through both Microsoft Proxy Server components (the Web Proxy and the WinSock Proxy). The logs (in either text format or database format) can then be used to generate reports of daily, weekly, or monthly activity.

Text File Logs

By default, Microsoft Proxy Server is configured to log all transaction information to text file logs. The default location of these files will be:

• Web Proxy Logs: c:\winnt\system32\w3plogs
• WinSock Proxy Logs: c:\winnt\system32\wsplogs

Normally, a new log file is created automatically every day, although this can be changed to weekly or monthly. A new log file can also be created when the current log file reaches a specific size.

In the Properties sheets of both the Web Proxy and WinSock Proxy services, there is a Logging tab. This tab is identical for both services (except for log file location values) and controls all logging settings for each server. Figure 13.1 shows the Logging tab of the Web Proxy.

Figure 13.1. The Logging tab of Web Proxy Properties.

Chapter 6, "Configuring Proxy Server," covers in detail all of the settings found on this tab. To ensure that logging is enabled for both proxy servers, make sure the Enable Logging check box is checked on both Logging tabs.

When a log file is created, its name is based on the current date. For example, a log file created on December 5, 1996 would have the file name: w3961205.log. Log files can be in one of two formats: regular and verbose. Regular logs are shorter format logs that do not contain all available data elements, and verbose logs contain a complete set of all available log data. Figure 13.2 shows a sample of a regular log file generated by the WinSock Proxy. The format of the logs files created by both proxy servers is identical.

Figure 13.2. A regular log file sample.

Figure 13.3 shows a sample of a verbose log file.

Figure 13.3. A verbose log file sample.

I have purposefully tried to access the same sequence of sites in both log files so you can see a side by side comparison (well, as close as possible since I did the first log file sample just before midnight and the second sample just past midnight).

As you can tell, each line of the log file is one record. Fields within the record are separated by commas. In the regular log file sample, the sequence of fields within a record is still the same; however, because a regular log is a reduced version of the verbose log, some of the fields are simply represented by a dash ( - ). This helps to cut down the size of the log files in busy proxy conditions. If you are the administrator for a network of 100 or more proxy users, you might find that daily logs files exceed 10 megs, depending on the amount of traffic that passes through the proxies.

Data Field Definitions

In order to understand the contents of the log file, the following is a definition of each field in the same order you will find in the log files themselves. Remember that log files from the Web Proxy and the WinSock Proxy are identical in format, and regular logs will not omit any data fields, just replace some information with a dash.

Figure 13.4 shows a more complicated log file example.

Figure 13.4. A full example of a Web Proxy Log file.

1. Client Computer IPÑThis field is the IP address of the client connecting to the proxy. When the Web Proxy performs an active caching session (where it goes out to the web on its own to refresh the contents of its cache) it will log an entry into the log as itself, and this field will be the IP of the Web Proxy server.
2. Client User NameÑIf the name of the user of the proxy client is known, it will be displayed in this field. If it is an anonymous client, the word "anonymous" will be used as a field value.
3. Client AgentÑThis field is the agent the client user is using to access the proxy server. With Web Proxy clients, the client application will send this information to the Web Proxy in the connection header. With WinSock Proxy clients, the WinSock client software on the workstation will determine the actual name of the program running and pass that along to the WinSock Proxy through the control channel for logging. This field also contains important client operating system information, separated from the agent name by a colon. With Web Proxy clients, this information may or may not be passed to the server in the connection header. With WinSock clients, this information is always passed to the server by the WinSock client software. An example of operating system information passed to the Web Proxy by a client may look like this: compatible; MSIE 3.0; Windows 95. Operating system information passed to the WinSock Proxy by the WinSock client software will look like this: 2:4.0, which is the version code for Windows 95. The following table details the operating system codes logged by the WinSock Proxy:

   Operating System Logged Code

   0:3.1 Windows 3.1

   0:3.11 Windows for Workgroups

   0:3.95 Windows 95 (connection made by a 16-bit client application.)

   1:3.11 Windows for Workgroups (connection by a client using the Win32s extensions.)

   2:4.0 Windows 95 (connection made by a 32-bit client.)

   3:3.51 Windows NT 3.51

   3:4.0 Windows NT 4.0

   A full example of the field value of this data bit logged by the Web Proxy might be might be: Mozilla/2.0 (compatible; MSIE 3.0; Windows 95). A full example of the field value of this data bit logged by the WinSock Proxy might be: WS_FTP32.EXE:2:4.0. The exact value of this field when logged by the Web Proxy will vary according to the header information sent to the Web Proxy by the client application making the proxy connection.

4. Authentication StatusÑThis field indicates if the client is connected with an authenticated connection. A Y value will indicate the client has been verified against the NT security database.
5. Date LoggedÑThe date when the proxy server created the log entry.
6. Time LoggedÑThe time when the proxy server created the log entry. This is given in 24-hour format.
7. Server NameÑThe name of the server that entered the log record. When verbose logging is selected, this will be WspSrv for the WinSock Proxy and it will be W3Proxy for the Web Proxy. When regular logging is select, this field will be a numeric value of 1 for the Web Proxy and a 2 for the WinSock Proxy.
8. Proxy NameÑThe name of the NT server running Microsoft Proxy Server. This will be the NetBios name given to the machine via the Network Control Panel.
9. Referring Server NameÑThis is currently a reserved field. In later releases of Microsoft Proxy Server it will be used to indicate the name of the downstream proxy server that referred the connection to the current proxy server. This will be helpful when dealing with cascaded proxy servers that operate in concert with each other. In the current version of Microsoft Proxy Server (1.0) this field will simply contain a dash.
10. Destination NameÑThis field indicates the domain name the client connected to through the proxy server. This is not always the name the client requested a connection to because some sites on the Internet perform automatic connection forwarding. If the information was pulled from the cache (only when dealing with the Web Proxy), this field will contain a dash.
11. Destination IP AddressÑThis field indicates the IP address the client connected to though the proxy server. As with the previous field, this information my simply be a dash if the data was obtained from the Web Proxy cache.
12. Destination PortÑThe TCP/IP port connection made between the proxy and the target site. With the Web Proxy, this will always be 80. The WinSock Proxy will log a wide range of ports for this field.
13. Processing TimeÑThe time it took (measured in milliseconds) the proxy server to obtain the requested information once the request was received from the client. The clock stops once the proxy server receives the result code from the target Internet site. If the information was retrieved from the Web Proxy cache, this field indicates how much time it took to locate the information and send it to the client.
14. Bytes SentÑThe number of bytes sent to the client by the proxy server. This field may be a dash if no data bytes where sent to the client. This field is only used by the Web Proxy. The WinSock Proxy only enters a dash in this field.
15. Bytes ReceivedÑThis field represents the number of bytes received by the proxy server from the client. This size is the size of the request line sent by the client to the proxy. As with the previous field, this field is not logged by the WinSock Proxy (it uses only a dash). If this field is a dash in a Web Proxy log, it may mean that the client sent no data or that it did not provide the size information.
16. Protocol NameÑIn Web Proxy logs, this field will be HTTP, FTP, Gopher, or Secure, depending on the type of protocol used by the client. In WinSock Proxy logs, this field will be the commonly used numeric protocol of the client connection (for example, 110 for SMTP connections).
17. TransportÑThe method of transport used between client and proxy. Web Proxy connections will always be TCP. WinSock Proxy connections will be either TCP, UDP, or IPX/SPX.
18. OperationÑIndicates the transport operation performed by the proxy server. The Web Proxy can log GET, PUT, POST, and HEAD. The WinSock Proxy can log Connect, Accept, SendTo, RecvFrom, and GetHostByName.
19. Object NameÑThis field indicates the name of the object retrieved by the Web Proxy. The WinSock Proxy always logs a dash in this field.
20. Object MIMEÑThis field is used only by the Web Proxy (the WinSock proxy simply logs a dash here). This field indicates the Multi-Purpose Internet Mail Extensions type for the retrieved object. This field may be a dash if the target Internet server is not defined or supported on the target server. The field can contain the following strings:

    MIME Type Definition

    application/x-msdownload Application

    image/gif GIF Image

    image/jpeg JPG Image

    multipart/x-zip ZIP Archive

    text/plain ASCII Text File

21. Object SourceÑOnly used by the Web Proxy, this field indicates where the object came from. The following value by be logged:

    Field Value Definition

    Unknown Proxy Server could not determine where the object originated.

    Cache Object found in cache.

    Rcache Object found on Internet. Objects was added to cache.

    Vcache Object found in cache. Object was verified against target object on Internet.

    NVCache Object found in cache but could not be verified against target object on Internet. Object was still returned to client.

    VFInet Object found on Internet. Object could not be verifed against source.

    PragNoCacheInet Object found on Internet. HTTP header indicates that the object should not be cached.

    Inet Object found on Internet. Object was not added to the cache.

22. Result codeÑThis field is the result code returned by the target Internet site concerning the retrieval of the object.. This field can be a wide range of values. The Web Proxy and the WinSock Proxy log different values for this field. In Web proxy log, values under 100 indicate Windows error codes. Values between 100 and 1000 are HTTP status codes. Values over 10000 are Wininet or WinSock error codes. The three most common codes logged by the Web proxy will be 200 (Successful Connection), 10060 (Connection Timed Out), and 10065 (Host Unreachable). In WinSock Proxy Logs, this field will be one of the following codes:

    Code Definition

    0 Successful Connection

1 Server Failure

2 Rejection by Proxy due to filtering

3 Network unreachable due to no DNS service available.

4 Host unreachable because no DNS entry could be found for the host.

5 Connection refused by target Internet site.

6 Unsupported client request (perhaps the client is using a non-compliant TCP/IP stack or the WinSock call is from a non-supported version.

7 Unsupported Address type.

Verbose Fields and Regular Fields

When regular logging is selected, some fields will simply be represented with a dash. Verbose logging will log all known data to all the previously listed fields. Regular logging will log only the following fields:

• Client Computer IP
• Client User Name
• Authentication Status
• Date Logged
• Time Logged
• Server Name
• Destination Name
• Destination Port
• Protocol Name
• Object Name
• Object Source
• Result Code

Reading the logs can sometimes be very confusing because it appears that the proxies sometimes do not log the correct information. The important thing to remember is to keep the order of the fields right, and pretty soon you'll be able to understand them correctly.

Log Cautions

Because the Web proxy runs as a sub-service of the WWW Server, you can turn off all WWW logging and let the Web Proxy log all connection information. Enabling the WWW log is a redundant action that will only eat up disk space.

Speaking of disk space, keep in mind that log files can grow to a very large size when there are many people accessing Microsoft Proxy Server. This is due to the fact that each object returned from the Internet or cache generates a record. Web pages can contain five, ten, or more objects. Every time such pages are accessed, a large number of log records are created. To make matters worse, some clients will issue the same command over and over again when they receive an error retrieving requested data. This can make the log files huge. It's an excellent idea to only keep a few days worth of logs active and archive the remaining ones you want to keep. If the log disk becomes full, Microsoft Proxy Server may stop operating.

Keeping a log will make sure that you as network administrator are maintaining a high security environment for your company. Without logging, your network users may be bogging down Internet access, and you'll have no way of knowing who or what is causing the problem.

Database Logging

If you have access to a database engine such as SQL Server or Access, you can use ODBC drivers to let Microsoft Proxy Server interface with these engines to log the same information that will be logged to text files. The benefit of logging to a database is that the data is already in an easily manipulated form and is often stored in a more compact manner.

When Microsoft Proxy Server is installed, it will prompt you to install ODBC drivers. The ODBC drivers for interfacing with SQL and Access are provided with Microsoft Proxy Server. If ODBC drivers are already installed on your NT server for some other purpose, you will not need to re-install them. ODBC drivers operate in a universal manner, and as such are not specific to any certain piece of software (except for the database they are designed to Interface with). ODBC was designed to be a generic interface method that applications could use to open a database to write or retrieve data without having to know the specifics of the database engine or format of the data files themselves. Applications make generic calls to the ODBC driver, and the ODBC driver knows how to perform the requested action with the database engine.

Preparing the Database

The first thing to do if you are planning on using a database to log Microsoft Proxy Server information is to prepare a database file to hold the data. This is done through the database itself. This should be a fairly straightforward one of creating the file with the required fields of sufficient length to hold any possible data. The following table details the exact field names and types Microsoft Proxy Servers needs in a table or database to correctly log information.

Table 13.1. Microsoft Proxy Server field names and types.
Field Name	SQL Data Type	Access Data Type	Length
ClientIP	varchar	text	50
clientUserName	varchar	text	50
ClientAgent	varchar	text	100
clientAuthenticate	char	text	5
LogTime	datetime	datetime	n/a
Service	varchar	text	25
ServerName	varchar	text	50
ReferredServer	varchar	text	100
DestHost	varchar	text	255
DestHostIP	varchar	text	50
DestHostPort	int	Long Integer	n/a
ProcessingTime	int	Long Integer	n/a
BytesSent	int	Long Integer	n/a
BytesRecv	int	Long Integer	n/a
Protocol	varchar	text	25
Transport	varchar	text	25
Operation	varchar	text	255
URI	varchar	text	255
MIMEType	varchar	text	25
ObjectSource	varchar	text	25
ResultCode	int	LongInteger	n/a

Once the table is constructed, the logging tab of the both proxy properties can be used to configure the proxies to log in to the correct database engine and pass log information. Chapter 6 covers setting up the logging tab in greater detail.

DSN stands for Data Source Name. It is a naming convention used by ODBC drivers 2.5 and above. When applications make DSN calls to an ODBC driver, those calls will include vital information such as the standard name of the ODBC driver being called, the name of the database to open, and the target server to contact. Without DSN, ODBC drivers will be unable to correctly open the target database and begin logging information. A database can be assigned a Data Source Name, and it is by this name that Microsoft Proxy Server will reference the target database. Make sure that the DSN you give to an SQL or Access database matches the DSN name you indicate on the logging tab of the proxy properties.

DSN is primarily configured through the ODBC Control Applet in the Windows NT Control Panel. Refer to the online help in that applet when configuring DSN for a particular database/engine.

Microsoft Proxy Server uses other logs for its own internal data. When Microsoft Proxy Server is installed, it logs its setup information to c:\mspsetup.log. This is a text log you can reference if setup did not go correctly.

When the WinSock Proxy client software is installed on workstations, it will create a log file called c:\mpcsetup.log. This log can be used to help track down installation problems workstations might be having with the WinSock Proxy client software.

The Windows Event Viewer found in the Administrative Tools folder is the best way of tracking down operational problems with Microsoft Proxy Server. Any application error Microsoft Proxy Server may be encountering will be logged through the Event Log. Each event related to Microsoft Proxy Server will be tagged with on of the following source names:

Event Source Name Definition

WebProxyServer Web Proxy Server generated
WebProxyLog Web Proxy logger generated
WebProxyCache Web Proxy cache generated
WinSockProxy WinSock Proxy Server generated
WinSockProxyLog WinSock Proxy Server logger generated
MSProxyAdmin Microsoft Proxy Server Administrative interface generated

SNMP is a communication protocol designed to allow remote control of a server or server application from an external workstation (or other NT server). In order for SNMP to work, the SNMP protocol must be installed on the server with the application that is to be remotely controlled, and any controlling server or workstation must have the SNMP protocol installed as well as SNMP client software. SNMP client software isn't cheap, nor is it simple to understand.

Most server applications that can be remotely controlled or monitored through the SNMP protocol will come with MIBs or Management Information Base files. MIBs are information files that tell SNMP client software how to interface with the server application they are designed for. SNMP client software will compile MIB files into part of their own code so they will understand how the target server application operates.

The Microsoft Proxy Server MIBs are not installed when Microsoft Proxy Server is installed. If you purchased Microsoft Proxy Server and have an installation CD, the MIBs can be found in the \Alpha\Perfctrs, \I386\Perfctrs, \Mips\Perfctrs, and Ppc\Perfctrs directories (for the four-system architectures that NT supports). The MIB for the Web Proxy service is w3p.mib. The MIB for the WinSock Proxy service is wsp.mib.

Compilation and usage instructions for SNMP control and monitoring is beyond the scope of this book. If you are interested in learning how to use the SNMP protocol and SNMP client software, you may have a hard time finding any demo or shareware clients. SNMP client software is not the kind of software companies put out for users use on a try before you buy basis.

For a large number of network administrators, using Microsoft Proxy Server will be the best way to control company-wide access to the Internet. Using the logging and SNMP monitoring capability of Microsoft Proxy Server, a network administrator can ensure that an Internet link will be used properly and efficiently.