Summary
How can a Hugging Face model be imported and deployed for inference in OCI Data Science?
The demonstration uses AI Quick Actions to register a model by its identifier, store its files in OCI Object Storage and deploy it on NVIDIA GPU infrastructure. A versioned bucket is created, and the Data Science terminal authenticates to Hugging Face through its CLI and an access token.
The model, container, bucket and target folder are then selected. Deployment configuration chooses the infrastructure and keeps default container arguments. Once available, the model is queried with parameters, and invocation options for CLI, Python and Java are reviewed.
The walkthrough ends by checking model files in Object Storage, where they remain available for other deployments.
Start
The presenter introduces importing Hugging Face models into OCI Data Science for projects using an endpoint or CLI. The walkthrough focuses on preparing storage, registering the model and deploying it for inference queries.
Bucket creation and versionation
An OCI Object Storage bucket is created with a name and versioning enabled. This destination will later store the files of the model registered through AI Quick Actions. Once the bucket is ready, a Data Science terminal opens.
Hugging Face Authentication with CLI and Token
The Data Science terminal starts authentication using the Hugging Face CLI. An access token is created in account settings with a name and options selected for the example. The token is then used in the terminal to finish signing in before model registration.
Model registration at AI Quick Actions
AI Quick Actions starts model registration using the identifier obtained from Hugging Face. The container, prepared bucket and target folder are selected. The fine-tuning option is enabled during registration, and the walkthrough waits for model files to download to the bucket.
Deployment of model at GPU
The registered model and NVIDIA GPU infrastructure are selected for deployment. Container arguments can be adjusted in the form, but the demonstration keeps default values. Deployment starts, and provisioning is allowed to finish before inference is tested.
Tests and invocation with CLI, Python and Java
Once deployment is available, the model is queried and test parameters are adjusted. Invocation options and examples for CLI, Python and Java are then shown. The walkthrough ends in Object Storage to check that model files have downloaded and are available for future deployments.