JavaScript is required to for searching.
Skip Navigation Links
Exit Print View
Copying and Creating Oracle Solaris 11.1 Package Repositories     Oracle Solaris 11.1 Information Library
search filter icon
search icon

Document Information

Preface

1.  Image Packaging System Package Repositories

2.  Copying IPS Package Repositories

3.  Providing Access To Your Repository

4.  Maintaining Your Local IPS Package Repository

Updating Your Local Repository

Checking and Setting Repository Properties

Customizing Your Local Repository

Serving Multiple Repositories Using Multiple Depot Server Instances

Depot Server Apache Configuration

Configuring Caching for the Depot Server

Cache Considerations for the Catalog Attributes File

Cache Considerations for Search

Running the Depot Server Behind a Web Proxy

Recommended Generic Apache Configuration Settings

Apache Configuration Examples

A Simple Prefixed Proxy Configuration

Multiple Repositories Under One Domain

Load Balanced Configurations

Complete Load Balanced Example

Depot Server Apache Configuration

This section discusses running the depot server behind an Apache web server instance to gain the following benefits:

Configuring Caching for the Depot Server

Minimal configuration is required to set up the depot server behind a caching proxy. With the exception of the catalog attributes file and repository search results, which are discussed below, all files served are unique and therefore safe to cache indefinitely if necessary. Also, all depot responses contain the appropriate HTTP headers to ensure files in the cache do not become stale by mistake.

See the Apache Caching Guide for more information about configuring Apache as a caching proxy.

Use the CacheRoot directive to specify the directory to contain the cached files. Make sure the specified directory is writable by the Apache process. No explicit error message is output if Apache cannot write to this directory.

CacheRoot /tank/proxycache

Apache allows you to enable caching for specific directories. You probably want your repository server to cache all the content on the server, as shown in the following directive.

CacheEnable disk /

Use the CacheMaxFileSize directive to set the maximum size of files to be cached. The Apache default of 1 MB might be too small for most repositories. The following directive sets the maximum cached file size to 1 GB.

CacheMaxFileSize 1000000000

Adjust the directory structure of the on-disk cache for the best performance with the underlying file system. In a ZFS dataset, multiple directory levels affect performance more than the number of files in one directory. Therefore, configure one directory level with a large number of files in each directory. Use the CacheDirLevels and CacheDirLength directives to control the directory structure. Set CacheDirLevels to 1. Set CacheDirLength to a value that results in a good balance between the number of directories and the number of files per directory. The value of 2 set below will generate 4096 directories. See the Apache Disk-based Caching documentation for more information.

CacheDirLevels 1
CacheDirLength 2

Cache Considerations for the Catalog Attributes File

The repository catalog attributes file (catalog.attrs) contains the current status of the repository catalog. This file can be large enough to warrant caching. However, this file becomes stale if the catalog of the back-end repository has changed. You can use one of the following two methods to address this issue.

Cache Considerations for Search

Searching a package repository generates custom responses based on the request. Therefore, search results are not well suited for being cached. The depot server sets the appropriate HTTP headers to make sure search results do not become stale in a cache. However, the expected bandwidth savings from caching are small. The following partial httpd.conf file shows how to specify not to cache search results.

<LocationMatch ".*/search/\d/.*">
        Header set Cache-Control no-cache
</LocationMatch>

Running the Depot Server Behind a Web Proxy

The pkg(5) depot server enables you to easily provide access to a repository in the local network or on the Internet. However, the depot server does not support serving multiple repositories under one domain name or sophisticated prefixes. To host multiple repositories under one domain name, run the depot server behind a web proxy. Running the depot server behind a web proxy can also improve the performance of the server by enabling load-balancing over multiple depots and enabling content caching.

The examples in this section use the Apache web server as the proxy software. The Oracle Solaris 11.1 OS includes the Apache web server in the web/server/apache-22 package and a basic httpd.conf file in /etc/apache2/2.2. The Apache web server is activated by enabling the svc:/network/http:apache22 service. See Apache HTTP Server Version 2.2 Documentation for additional information.

You should be able to apply the principles shown in these examples to any proxy server software.

Recommended Generic Apache Configuration Settings

The following settings affect performance and security.

Enable the Apache DEFLATE filter.

HTTP clients can tell the server that they accept compressed data in an HTTP request. Enabling the Apache DEFLATE filter can dramatically reduce the over-the-wire size of metadata such as catalogs and manifests. Metadata such as catalogs and manifests often compress 90%.

AddOutputFilterByType DEFLATE text/html application/javascript text/css text/plain
Do not decode encoded forward slashes.

Packages can contain URL encoded forward slashes. To make sure these forward slashes are not interpreted as directory delimiters, instruct Apache to not decode them.

AllowEncodedSlashes NoDecode

Note - Omitting this setting very negatively impacts search functionality.


Allow more pipelined requests.

Increase the MaxKeepAliveRequests value to allow clients to make a larger number of pipelined requests without closing the connection. The Apache default of 100 is too low.

MaxKeepAliveRequests 10000
Set the maximum wait time for response.

The proxy timeout sets how long Apache waits for the back-end depot to respond. For most operations, 30 seconds is satisfactory. Searches with a very large number of results can take significantly longer. You might want a higher timeout value to accommodate such searches.

ProxyTimeout 30
Disable forward proxying.

Make sure that forward proxying is disabled.

ProxyRequests Off

Apache Configuration Examples

This section illustrates multiple repository, non-load-balanced, and load-balanced setups.

A Simple Prefixed Proxy Configuration

This example shows the basic configuration for a non-load-balanced depot server. This example connects http://pkg.example.com/myrepo to internal.example.com:10000.

See Serving Multiple Repositories Using Multiple Depot Server Instances for instructions about setting other properties you need that are not described in this example.

You should configure the depot server with a pkg/proxy_base setting that names the URL at which the depot server can be accessed. Use the following commands to set the pkg/proxy_base:

$ svccfg -s pkg/server add repo
$ svccfg -s pkg/server:repo "setprop pkg/proxy_base = astring: http://pkg.example.com/myrepo"
$ svcadm refresh pkg/server:repo
$ svcadm enable pkg/server:repo

The pkg(5) client opens 20 parallel connections to the depot server when performing network operations. Make sure the number of depot threads matches the expected connections to the server at any given time. Use the following commands to set the number of threads per depot:

$ svccfg -s pkg/server:repo "setprop pkg/threads = 200"
$ svcadm refresh pkg/server:repo
$ svcadm restart pkg/server:repo

Use nocanon to suppress canonicalization of URLs. This setting is important for properly working search. Also, limit the number of back-end connections to the number of threads the depot server provides. The following partial httpd.conf file shows how to proxy one depot server:

Redirect /myrepo http://pkg.example.com/myrepo/
ProxyPass /myrepo/ http://internal.example.com:10000/ nocanon max=200

Multiple Repositories Under One Domain

The most important reason to run the depot server behind a proxy is to easily run several repositories under one domain name with different prefixes. The example from A Simple Prefixed Proxy Configuration can be easily extended to support multiple repositories.

In this example, three different prefixes of one domain name are connected to three different package repositories:

The pkg(5) depot server is an SMF managed service. Therefore, to run multiple depot servers on the same host, simply create a new service instance:

$ svccfg -s pkg/server add repo1
$ svccfg -s pkg/server:repo1 setprop pkg/property=value
$ ...

Like the previous example, each depot server runs with 200 threads.

Redirect /repo_one http://pkg.example.com/repo_one/
ProxyPass /repo_one/ http://internal.example.com:10000/ nocanon max=200

Redirect /repo_two http://pkg.example.com/repo_two/
ProxyPass /repo_two/ http://internal.example.com:20000/ nocanon max=200

Redirect /xyz/repo_three http://pkg.example.com/xyz/repo_three/
ProxyPass /xyz/repo_three/ http://internal.example.com:30000/ nocanon max=200

Load Balanced Configurations

You might want to run depot servers behind an Apache load balancer. This example connects http://pkg.example.com/myrepo to internal1.example.com:10000 and internal2.example.com:10000.

Configure the depot server with an appropriate proxy_base setting as shown in A Simple Prefixed Proxy Configuration.

Limit the number of back-end connections to the number of threads each depot is running divided by the number of depots in the load-balancer setup. Otherwise, Apache opens more connections to a depot than are available and they stall, which can decrease performance. Specify the maximum number of parallel connections to each depot with the max= parameter. The example below shows two depots, each running 200 threads. See A Simple Prefixed Proxy Configuration for an example of how to set the number of depot threads.

<Proxy balancer://pkg-example-com-myrepo>
        # depot on internal1
        BalancerMember http://internal1.example.com:10000 retry=5 max=100

        # depot on internal2
        BalancerMember http://internal2.example.com:10000 retry=5 max=100
</Proxy>

Redirect /myrepo http://pkg.example.com/myrepo/
ProxyPass /myrepo/ balancer://pkg-example-com-myrepo/ nocanon

Complete Load Balanced Example

The following example includes all the directives you need to add to the httpd.conf file for a repository server hosting a load-balanced and a non-load-balanced depot server setup.

In this example, two different prefixes of one domain name are connected to three different package repositories:

AddOutputFilterByType DEFLATE text/html application/javascript text/css text/plain

AllowEncodedSlashes NoDecode

MaxKeepAliveRequests 10000

ProxyTimeout 30

ProxyRequests Off

<Proxy balancer://pkg-example-com-repo_one>
        # depot on internal1
        BalancerMember http://internal1.example.com:10000 retry=5 max=100

        # depot on internal2
        BalancerMember http://internal2.example.com:10000 retry=5 max=100
</Proxy>

Redirect /repo_one http://pkg.example.com/repo_one/
ProxyPass /repo_one/ balancer://pkg-example-com-repo_one/ nocanon
Redirect /repo_two http://pkg.example.com/repo_two/
ProxyPass /repo_two/ http://internal.example.com:20000/ nocanon max=200