Implementing IT Chargeback

In UNIX and Windows

 

 

 

UniSolutions Associates LLC

http://www.unisol.com

949-488-3960

1. Background

System usage tracking and chargeback has long been a standard practice on mainframes, however, its use elsewhere in the enterprise has been largely limited to government agencies, contractors and local governments.  Server consolidation projects however, where many smaller servers are replaced by a larger multiprocessing server, have brought up the need to track computer resource usage and equitably allocate expenses to the various departments and projects using common servers (whether they run UNIX, Linux or Windows).  In fact, with today's larger multiprocessor computers, implementing resource tracking and chargeback is necessary for any data center that wants to accurately track system usage and turn the IT organization into a profit center instead of a cost center or simply an overhead department.

2. Introduction

Early implementations of chargeback systems were intended for mainframe class computers, used by multiple departments for many different purposes. Services provided by IT staff included:

• Software Support,
• System Administration,
• Help Desk functions,
• File Backups,
• Communications.

In PC environments, users migrate some (or most) of their work to their desktop, however they still need services such as:

• Central Databases,
• Installation Support,
• Help Desk, and
• Corporate E-mail.

Users in workstation environments are similar to PC users because they can control a larger part of their environment, but they still require central services in order to remain productive. They may need services such as:

• Software Support,
• System Administration,
• Help Desk functions,
• File Backups,
• Network Connectivity.

IT organizations today, usually support several large servers (perhaps some shared servers as a result of server consolidation projects) as well as some smaller special-purpose servers (mail servers, print servers, etc…), along with a mix of workstations, PC’s and other computing devices (routers, firewalls, etc…).   Thus, the IT organization is still responsible for the same kind of services as with a large mainframe, although on a different scale. Now an administrator supports dozens and maybe hundreds of computers (not just dozens or hundreds of users). While some of those computers were purchased using the user's own budget, support services are still provided by the IT organization and must be recouped.

Additionally, the purpose of a chargeback strategy is not only to recoup IT costs, but to also use the data produced by the chargeback system to forecast hardware needs and maintain a balance between supply and demand.

3. Defining a chargeback strategy

Traditionally, IT hardware, software and support costs were overhead to be borne by the departments that used the hardware and services provided by IT. The resources necessary to run the IT organization were set at departmental meetings depending on the relative well-being of the departments and the company. In these cases it was not uncommon that demand far exceeded supply, sometimes negatively contributing to workers' productivity. However, when a chargeback strategy is applied to IT resources, keeping track and responding to changes of supply and demand becomes easier, allowing the IT organization to add resources when demand increases thus reducing or eliminating user complaints due to insufficient resources, and quite possibly mitigate any adverse business conditions caused by the lack of resources.

Operating systems generally provide some logging and auditing functionality that can be used to track system resource usage. Standard UNIX accounting tracks such things as logins and logouts and resources consumed by each process. This is inadequate for the needs of most organizations though, because it is impossible to generate reports at the project or department level. It also does not fill the needs of government contractors and government agencies which must keep track of system resource usage by project. Earlier versions included modifications to the UNICOS kernel for project accounting on CRAY systems as well as SRI modifications for Pyramid systems.

Event Log monitoring on Windows systems can also provide some data on logins and application image time. However the available data is again inadequate to satisfy the needs of an organization since more data is usually required and no project or department level information is available.

There are several ways though that the data available on the different operating systems can be used and supplemented in order to provide a complete and accurate picture of IT resource consumption when implementing a chargeback system. UNISOL JobAcct intelligently uses data provided by various operating systems in order to provide complete chargeback solutions in IT organizations utilizing UNIX, Linux and Windows systems. It is widely used by U.S. Government Agencies and government contractors, as well as by large corporations and universities that want to track system usage.

There are several factors to consider when defining and implementing a chargeback strategy.  A key consideration is keeping the chargeback strategy simple by reducing billable items to as few cost pools as possible in order to keep control of the chargeback process, and, to charge only for the resources whose consumption can be accurately tracked or estimated.  Another key consideration is the quality of the information provided by the chargeback process.  The information must be reliable, relevant and material, before it can be relied upon in the decision-making process of the IT organization.

4. Common Chargeback Strategies

There are several options when developing a chargeback scheme. They are explored below:

4.1.                  Simple allocation by headcount

Under this scheme, the total IT costs are added up and then divided by the number of employees. Then each department is charged depending on the number of employees under it. This would be fine if all employees consumed resources equally, but creates inequities when they don't.

4.2.                  Estimated allocation by department

This scheme attempts to cure the inequities created by the first method, however, some departments will feel that they are being overcharged because they don't know that they actually consume the share of the resources for which they are actually billed.

4.3.                  Utilization billing

Under this scheme, departments get billed for what they actually use, overcoming the objections common with the other two schemes.  The key to the implementation of a utilization billing system is the accurate tracking of resource consumption across all IT resources.  Limiting chargeback to a subset of IT resources can affect the shift of resource consumption and distort usage patterns, especially when user organizations can shift resource usage to untracked resources.

4.4.                  Fixed cost utilization billing

This scheme is a variation of the utilization billing scheme and attempts to proportionately allocate the fixed costs of running an IT organization depending on the consumption of resources as indicated by the utilization billing method.

From the options presented above, either of the first two schemes can be satisfied by using a good spreadsheet and some (or a lot of) imagination, while UNISOL JobAcct can help with the last two schemes.

When selecting an allocation chargeback scheme additional considerations must be made in order to handle unplanned or not budgeted expenses by specifying procedures for such situations, and ensuring that the budget planning process has customer approval.  If the cost allocation though is not equitable users may be tempted to device their own solution, thus eventually marginalizing the IT organization. Allocation based chargeback schemes work best with a small and/or static user base with a predictable workload.

Usage based chargeback schemes are usually required for organizations with a large or dynamic user base, organizations with unpredictable workloads, or where mandated by government regulations or customer agreements.

5. Utilization Chargeback Benefits

The utilization billing methods described in this paper help transform the IT organization from an overhead department to a for profit center, by allowing the IT organization to  hold business units accountable for the resources they consume, foster cost efficiency throughout the organization, and, to plan and budget for any changes (especially increases) in service demands.

 

IT organizations that can accurately track current resource usage and forecast future demand of technology and service needs are better prepared to address any such needs because they can control their own budgets to efficiently address increased demand and new technology needs.  Chargeback data must be distributed to customers for regular review in a format easy to understand, and the data must be repeatable and controllable, and the charges must comport to the real cost of the services provided. Data obtained in the chargeback process can identify obsolete, unused or money-losing services, allowing the IT organization to make informed decisions when developing a long-term plan for cost savings and opportunities for performance improvements. Therefore, chargeback allows the IT organization to transform itself into a technology enabler, capable of taking advantage of new technologies and allowing executive management to make a business case that considers the long-term objectives of technology investments.

 

In addition, utilization chargeback allows the IT organization to better understand resource usage patterns for the various projects within the enterprise and to provide accurate capacity guidance when new projects are launched, thus allowing new projects the resources required to succeed.  Knowledge gained from chargeback helps the IT organization create a framework for tying technology investments to business objectives for performance improvements.

 

In summary, IT chargeback can help an organization cut costs, improve efficiencies and improve investment decision making by facilitating the better allocation of IT infrastructure with the areas of business benefit, elevating the IT organization from cost center to business enabler.

6. Implementing chargeback with JobAcct

UNIX systems already generate most of the raw data required to adequately track system usage. UNISOL JobAcct takes care of the rest (e.g. printer charges, miscellaneous IT charges, database charges), and performs accounting by project and combines data for all of the servers and workstations on the network into a single report.

On Windows systems some resource usage data is available in the Event Log: login and logout time as well as application image time (wall clock time of when an application is run). UNISOL JobAcct can use this data for the chargeback application if this is a chargeable category for an organization. However, application image time data does not take into account whether any CPU time is consumed if the user starts an application but performs no other activity. Thus, UNISOL JobAcct also collects CPU and memory usage data in order to accurately charge for system usage (since connect time is already a chargeable category). The Event Log is also a source for "pages printed" information. This data is used and summarized along with other system data by JobAcct.

UNISOL JobAcct also generates any other data required to adequately track system usage at the project level (through the installation of some Windows services). Furthermore, UNISOL JobAcct for Windows (just like its UNIX counterpart) performs accounting by project and combines all of the machines on the network in a single report. Report generation can take place on UNIX or Windows, where data from UNIX and Windows machines can be combined into one report.

Extending the implementation of resource tracking and chargeback inside the database is required in order to completely track database application activity, especially when charging back for database server use, or when database applications are a large part of the application mix.

Tracking database application usage is generally accomplished by implementing triggers in the database itself in order to accurately track database resource usage at the session level. Some database tracking applications perform sampling, which can adversely affect database performance and waste CPU resources. Tracking database usage at the session level can be accomplished without being overly intrusive and adversely affecting database performance.  JobAcct tracks resource usage in the Oracle, Sybase, DB2 and SQL Server databases. The database accounting data is available for inclusion in all of JobAcct's standard reports.

6.1.                  UNISOL JobAcct Features

UNISOL JobAcct performs actual cost accounting and chargeback for each user, group, project and cost center, or, proportional chargeback in order to allocate a set of fixed costs at a project/department level. Accounting can be restricted to specific commands or can include all commands, with individual charges reported for each tracked resource type.

Charges are reported for the following resource types:

• Session connect time
• Disk space usage
• CPU usage
• Memory usage
• Pages printed
• Disk I/O operations
• Miscellaneous charges
• Database activity
• Session time
• CPU usage
• Disk I/O
• Memory usage
• Net I/O

Billing reports can include addresses for direct mailing to users and their organizations. Reports can be easily added or modified with awk, Perl or other report writing programs. Delimited file reports can also be generated and easily reformatted for use with databases or other financial programs.

Accounting time periods are specified for each day of the week and different rates can be applied for each time period (by project number):

• Prime
• Non-Prime
• Reduced

A new charge number can be set at any time during a user's session with the "charge" command. If the user never sets a charge number, then the user's default is applied. System-wide or personal aliases can be used to refer to chargeback numbers.

Users can "review" their own charges, up to the last available accounting summary, for each billing period.

Flexible distribution of disk charges among many chargeback/project numbers.

Summary reporting for a top-down view of the overall system or project level accounting charges, including percentage information (of both resource usage and revenue sources) for each chargeable field, and generates pie-charts of system usage at the chargeback number level.

Direct account billing and budget tracking and reporting (by user and project).

An optional performance monitoring module collects system activity and reports system performance for specific resources in both tabular and graphical form.

6.2.                  UNISOL JobAcct Reports

Chargeback Reports (including addresses for mailing):

• Billing Reports (record per page)
• per User, Group, Project or Cost-Center Reports
• Summary Reports

Detailed Reports

• Detailed Process Activity
• Per-User Command Usage
• Login Accounting - similar to last
• Database Accounting - Detailed Session Report

Summary Reports

• Bar-Charts and Pie-Charts
• Top-Level Report
• Top-Level Proportional Report
• Budget Status Reports

Custom Reports

Generate custom reports by combining various reporting options and/or modifying the supplied reporting scripts.

Database Reports

Database reports can be produced together with the standard billing and summary system usage reports or can be produced separately. Detailed database usage reports by session are also available.

7. Using UNISOL JobAcct

Product configuration can be accomplished by modifying and maintaining the relevant configuration files directly (either modifying them directly with an editor or by generating them from a database), or, by using the provided JobAcct Application Management Interface. JobAcct's Application Management user interface aids the administrator in configuring the product by specifying the available projects, user defaults and system defaults, and allows the generation of various usage reports.

A command level interface allows the generation of reports from the command line or from shell scripts, providing flexibility whenever required to interoperate with other corporate accounting systems and/or corporate databases.

8. Implementation Notes

UNISOL Job Accounting has been implemented for organizations which provide system services and system access to other organizations either within or outside of the organization and/or company. In addition, JobAcct has been implemented for organizations which require the ability to distribute each user's work across multiple projects while maintaining reasonable controls over the way system resource usage is reported over several different projects. Finally, JobAcct has been implemented in such a way as to cause no disruption of other system services provided, and without requiring kernel modifications.

Nightly intermediate reports provide for flexible reporting/billing periods by combining any number of nightly summaries to produce custom billing reports for users, groups and projects (which can vary in the number of days included in each report, users or summaries by groups and projects in addition to the individual user and project breakdown).

Since disk accounting is(/may be) scheduled nightly, the default billing report disk usage is based on the average disk space used over the entire billing period. An alternate method of totaling disk space usage samples is also available. The user can allocate percentages of each directory to many chargeback numbers.

Chargeback commands are usually executed by the administrator, and generate login and process accounting reports as well as usage and billing reports (from detailed to top level summaries).

Some maintenance commands, usually invoked by cron or other daemons, generate usage summary data which can be used later by JobAcct to produce summary usage, billing and budgeting reports.

UNISOL JobAcct provides the flexibility to charge for virtually any type of service provided to the user. The miscellaneous charge interface can be used to charge users for any provided service by allowing its use within batch scripts to automatically charge for provided services.

 

Contact:

        UniSolutions Associates LLC

        33586 Via Lagos

        Dana Point, CA. 92629

        Phone: 949-488-3960

        FAX: 949-443-5354

        WWW: http://www.unisol.com

        Email: info@unisol.com