RADICAL-Pilot 0.23 Documentation¶
RADICAL-Pilot is a Pilot Job system
written in Python. It allows a user to run large numbers of computational
tasks (called ComputeUnits
) concurrently on one or more remote
ComputePilots
that RADICAL-Pilot can start transparently on a multitude of
different distributed resources, like HPC clusters and Clouds.
In this model, the resource is acquired by a user’s application so that the
application can schedule ComputeUnits
into that resource directly, rather
than going through the system’s job scheduler. In many cases, this can
drastically shorten overall exeuction time as the individual ComputeUnits
don’t have to wait in the system’s scheduler queue but can execute directly
on the ComputePilots
.
Mailing Lists
- For users: https://groups.google.com/d/forum/radical-pilot-users
- For developers: https://groups.google.com/d/forum/radical-pilot-devel
Contents:¶
- 1. Introduction
- 2. Installation
- 3. Using Local and Remote HPC Resources
- 4. Data Staging
- 5. Examples & Best Practice
- 5.1. Getting Started
- 5.2. Error Handling Strategies
- 5.3. Disconnecting and Reconnecting
- 5.4. Launching Remote / HPC ComputePilots
- 5.5. Executing Multiple Commands in a Single ComputeUnit
- 5.6. Executing Multicore / Multithreaded ComputeUnits
- 5.7. Application Control Flow with Callbacks
- 5.8. Mandelbrot set
- 5.9. K-Means Clustering
- 6. Tutorial
- 7. Unit Scheduler
- 8. Testing
- 9. Benchmarks
- 10. Frequently Asked Questions
- 10.1. Q: I see the error “OperationFailure: too many namespaces/collections”
- 10.2. A: Try Cleaning-up MongoDB.
- 10.3. Q: I see the error “Permission denied (publickey,keyboard-interactive).” in AGENT.STDERR or STDERR.
- 10.4. A: Set up password-less, intra-node SSH access.
- 10.5. Q: On Gordon I see “Failed to execvp() ‘mybinary’: No such file or directory (2)”
- 10.6. A: You need to specify the full path of the executable as mpirun_rsh is not able to find it in the path
- 10.7. Q: How many concurrent RADICAL-Pilot scripts can I execute?
- 10.8. Q: Why do I get errors from setuptools when trying to use a virtualenv?
- 10.9. A: Most likely because an upgrade of pip or setuptools failed
- 11. Developer Documentation
- 12. API Reference
- 13. Release Notes
- 13.1. 0.23 Release 2014-12-13
- 13.2. 0.22 Release 2014-12-11
- 13.3. 0.21 Release 2014-10-29
- 13.4. 0.19 Release September 12. 2014
- 13.5. 0.18 Release July 22. 2014
- 13.6. 0.17 Release June 18. 2014
- 13.7. 0.16 Release June 17. 2014
- 13.8. 0.15 Release June 12. 2014
- 13.9. 0.14 Release June 11. 2014
- 13.10. 0.13 Release May 19. 2014
- 13.11. 0.12 Release May 09. 2014
- 13.12. 0.11 Release Apr. 29. 2014
- 13.13. 0.10 Release Apr. 29. 2014
- 13.14. 0.9 Release Apr. 16. 2014
- 13.15. 0.8 Release Mar. 24. 2014
- 13.16. 0.7 Release Feb. 25. 2014
- 13.17. 0.6 Release Feb. 24. 2014
- 13.18. 0.5 Release Feb. 06. 2014
- 13.19. 0.4 Release
- 13.20. 0.1.3 Release
- 13.21. 0.1.2 Release