Skip to content

New Multi-Enclave Allocation Service #685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
eraykaradag wants to merge 17 commits into
base: main
Choose a base branch
from
Open

New Multi-Enclave Allocation Service #685

eraykaradag wants to merge 17 commits into from

Conversation

@eraykaradag
Copy link

@eraykaradag eraykaradag commented May 13, 2025

Old PR: eraykaradag#1

This PR introduces a new allocator service that enables granular resource allocation for multiple enclaves, implemented in Rust for improved error handling and user experience.

Changes

  • Implemented per-enclave resource allocation using sysfs
  • Enhanced configuration format supporting multiple enclave definitions
  • Maintained backward compatibility with existing configurations

New Configuration File Format
allocator.yaml:

- memory_mib: 1024
  cpu_count: 2  
- memory_mib: 512
  cpu_pool: 2,6

Key features:

  • Supports concurrent cpu_count and cpu_pool usage
  • All resources are allocated within the same NUMA node
  • Prevents resource overallocation by nitro-cli

Testing Done

  • Verified backward compatibility
  • Tested on ARM and x86_64 instances
  • Validated on multi-NUMA instances
  • Added unit tests with sysfs mocking
  • Created CTS test case for user-reported issues
  • Enhanced MultiSanity payload tests

Notes

  • No changes required to nitro-cli resource manager
  • Improved error messages guide users through allocation failures
  • Maintains strict resource isolation between enclaves

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@eraykaradag eraykaradag force-pushed the allocator-approval branch 4 times, most recently from a2ccc54 to ed5a91d Compare May 13, 2025 17:36
@eraykaradag
Backward compatibility support
4b8f205
@eraykaradag
Fixed the feedback
6d429b7
@eraykaradag
Allocator: Added Unit tests and Documentation for functions
3845762 
I have added unit tests to cover general functionality of allocator service. Tried to mock SysFs files to check if it will work or not. Also i have added documentation for functions to explain general functionality.
@eraykaradag
allocator/src/configuration.rs: Changing magic number with constant
fe8b8e3
@eraykaradag
allocator: Polished error messages and fixed serde error for deserial…
9bbb97b 
…izing strings

I changed error type with anyhow. Tried to add more information to error messages. Also found an error with serde with deserializing numbers into string when user provides pool of cpus. fixed it with implementing separate deserializer logic.
@eraykaradag
Makefile: Changed old allocator script with new allocators binary
1cd35b7 
Changed makefile to build allocator as a separated binary. Since binary and the bash script has the same the allocator.service will pick up the new one.
@eraykaradag
allocator/src/configuration.rs: fix: Sanity check added when looking …
a03f4b3 
…for NITRO_CLI_INSTALL_DIR env variable. It fails to read config file if env variable is not exist.

If env variable not exist then allocator will try to find the config file in / directory. Tried to build everything manually i bumped into this error in amazon linux and ubuntu instances. Also legacy allocator has the same mechanism but it was not failing. it was failing in Rust so i had to add a sanity check.
@eraykaradag
build-docs: allocator: Added the new allocator into all build flows a…
f33a78c 
…nd also every documentation to guide how to build the allocator

Since the new allocator is a binary i put it's build flow with the nitro-cli and vsock-proxy. Also added to the how to install and setup nitro-cli documentations.
@eraykaradag
Formatting allocator project with cargo fmt
466fa8a
@eraykaradag eraykaradag force-pushed the allocator-approval branch from ed5a91d to 466fa8a Compare May 13, 2025 17:48
@eraykaradag eraykaradag marked this pull request as ready for review May 14, 2025 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@eraykaradag @mariusknaust