The Prognostic Lab | Computer Science Department | University of Pittsburgh


Pisces is a co-kernel architecture designed to allow multiple specialized OS/R instances to execute concurrently on the same local node. Pisces enables the decomposition of a node's hardware resources (CPU cores, memory blocks, and I/O devices) into partitions that are fully managed by independent OS kernels/system software stacks. Using Pisces, a local compute node can initialize multiple Lightweight Kernel instances (based on the Kitten LWK) as co-kernel enclaves executing alongside an unmodified Linux host OS. Furthermore, by leveraging Palacios support, virtual machine instances can be created on top of these co-kernels as well.


The Pisces framework dynamically decomposes a node’s hardware resources into multiple isolated partitions, each capable of supporting a fully independent and isolated OS environment. Each OS instance is referred to as an enclave and is dynamically constructed based on the runtime requirements of an application.

In order to avoid modifications to the host Linux environment, Pisce relies on the ability to offline resources in modern Linux kernels. The offline functionality allows a system administrator to remove a given resource from Linux’s allocators and subsystems while still leaving the resource physically accessible. By off-lining resources, pisces partitions Hardware by dynamically removing resources such as CPU cores, memory blocks and PCI devices from a running Linux kernel. Once a resource has been off-lined, a running co-kernel is allowed to assume direct control over it. In this way, both Linux and Pisces co-kernels are able to manage discontiguous sets of resources explicitly assigned to them.

Some of the design features provided by Pisces:

  1. Elimination of cross OS dependencies.
  2. Internalized management of I/O.
  3. Cross enclave communication limited to explicit user-level shared memory segments.
  4. Usage of virtualization techniques to provide missing OS features.
Currently, Pisces supports co-kernel architectures based on the Kitten Lightweight Kernel and Palacios Virtual Machine Monitor.


(1) Enclave Resource Assignment
Download Pisces


Related Publications

HPDC [2015] (pdf) J. Ouyang, B. Kocoloski, J. Lange and K. Pedretti,
Achieving Performance Isolation with Lightweight Co-Kernels,
Proceedings of the 24th International ACM Symposium on High Performance Parallel and Distributed Computing, (HPDC 2015)
ROSS [2015] (pdf) B. Kocoloski, J. Lange, H. Abbasi, D. Bernholdt, T. Jones, J. Dayal, N. Evans, M. Lang, J. Lofstead, K. Pedretti, P. Bridges,
System-Level Support for Composition of Applications,
Proceedings of the 5th International Workshop on Runtime and Operating Systems for Supercomputers, (ROSS 2015)

All content and images © 2015 Jack Lange