![]() Jenkins-CI workflow for parallel image processing using CellProfiler on the Linux cluster. Instrument data shares act as the final secure repository for important analysis data. The instruments data shares store large data/image sets, shared between multiple OS systems (Windows/Linux). The projects build history stores build metadata, transient analysis data, and reusable components such as pipelines and image lists. Highperformance parallel computing tasks (such as image processing) can be easily integrated into Jenkins-CI projects using standard SSH access provided by the SSH plugin. Installed Jenkins-CI plugins and local scripts and applications execute on the Jenkins-CI server and provide an extensible set of data management and processing functions. A Jenkins-CI project configuration template defines the parameters, environment, and actions executed by a project build and drives the generation of the user interface. A typical Jenkins-CI installation (shown in the center) integrates computational resources (blue rectangles) and local and remote data (green file folders) and makes them accessible to end users via a standard web portal. Limitations with Jenkins-CI (primarily around the user interface) were addressed through the selection of helper plugins from the Jenkins-CI community.Īrchitecture of Jenkins-CI configured as a scientific data-processing platform. Pipelines and managed data are annotated to facilitate collaboration and reuse. Imaging pipelines developed using the desktop CellProfiler client can be managed and shared through a centralized Jenkins-CI repository. The platform is web-accessible, facilitates access and sharing of high-performance compute resources, and automates previously cumbersome data and image-processing tasks. Using Jenkins-CI, we integrated CellProfiler, an open-source image-processing platform, with various HCS utilities and a high-performance Linux cluster. Jenkins-CI provides numerous plugins for standard compute tasks, and its design allows the quick integration of external scientific applications. We describe how Jenkins-CI, an “off-the-shelf,” open-source, continuous integration system, is used to build pipelines for processing images and associated data from high-content screening (HCS). Scientific data integration and pipelining facilitate standardized data processing, collaboration, and reuse of best practices. Building integrated, scalable, and robust computational workflows for such applications is challenging but highly valuable. Note: All fields described in the sections below (after the properties file example) are required unless explicitly described as “optional.” In your own properties file, you would replace values surrounded with with the relevant information.High-throughput screening generates large volumes of heterogeneous data that require a diverse set of computational tools for management, processing, and analysis. Contact us on the CellProfiler forums if you need help with this. Note: CPA 2.0 is not compatible with properties files from CellProfiler Analyst version 1.0, but the two formats may be easily converted by hand. We suggest using Notepad on Windows, TextEdit on Mac OS, and Emacs on Linux. Note: When editing the properties file, it is important to use an editor that is capable of saving plain text. Settings that require a file path may be specified either as absolute or relative to the directory that the properties file is found in. Lines that begin with a # are ignored by CPA and may be used for comments. ![]() Otherwise, you can create one manually, referring to the Properties_README or the example provided below as a template.Įach setting in the properties file is stored on a separate line in the form field = value(s), and the order of the settings is not important. If you use CellProfiler to produce the data to be analyzed in CPA, you can automatically generate a nearly complete properties file with, using the ExportToDatabase module. It is selected and loaded upon startup of CPA. This file can be stored anywhere on your computer. The properties file is a plain text file that contains the configuration information necessary for CPA to access your data and images. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |