Summary
How can CodeLlama 13B be adapted to SQL and PL/SQL examples with AI Quick Actions?
The demonstration uses OCI Data Science to prepare a JSONL dataset, select a base model and configure fine-tuning. Files and results are stored in a versioned OCI Object Storage bucket. Model version, GPU infrastructure, network and parameters such as epochs are selected.
The job is monitored, model provenance and parameters are reviewed, and an endpoint is deployed. Python invocation includes Server-Sent Events support and authentication through ADS and Resource Principal.
Test Your Model uses a question from the training dataset itself to inspect the generated SQL and column names. An extra code-based implementation converts JSONL to CSV and separates training and chat applications. The narrator explains that this final test response still needs adjustment.
Start
AI Quick Actions in OCI Data Science is introduced for adapting pretrained models through fine-tuning. The walkthrough uses CodeLlama with a custom code-generation dataset before deploying the adjusted model and testing invocation.
Preparation of the JSONL dataset for SQL and PL/SQL
A dataset of SQL and PL/SQL questions is prepared in JSONL format. The presenter explains that lines can be processed independently. The file is loaded into Data Science or directly into Object Storage as input for model fine-tuning.
Object Storage bottle with versionation
An OCI Object Storage bucket with object versioning enabled is prepared as a prerequisite. This destination is selected in the fine-tuning flow to store the adjusted model results.
CodeLlama 13B finetuning configuration
CodeLlama 13B is selected in AI Quick Actions, and fine-tuning opens. The dataset location and file are chosen, a model version label is defined, and the prepared bucket is selected for results before infrastructure configuration.
GPU infrastructure and training parameters
GPU infrastructure and the same VCN as the Data Science environment are selected. Log integration is presented for monitoring, and parameters such as epochs are configured. The flow is then reviewed, and the job is submitted through AI Quick Actions.
job monitoring and execution review
View Console opens the OCI job status. Once execution finishes, model provenance and execution parameters are inspected. The narrator reports approximately 10 minutes for fine-tuning in this particular run. The review connects the resulting model to its training configuration.
Retirement of the adjusted model
AI Quick Actions deploys the adjusted model on selected GPU inference infrastructure. The console tracks deployment status and work requests. The narrator reports approximately 20 minutes for this deployment before moving on to the endpoint.
Invocation of endpoint from Python
Invocation options display the endpoint and a Python example. The code is copied into a Data Science file, and Server-Sent Events support is prepared. This receives server messages as a stream of updates for the application.
Authentication with ADS and Resource Principal
The Python example changes authentication to ADS with Resource Principal. The application invokes the model using the permissions of the OCI resource where it runs. The example is then executed and presented as a starting point for adapting integration to other applications.
Test model with dataset questions
Test Your Model receives a question copied from the training dataset itself. The SQL response and column names are compared with the loaded examples. This checks an adjusted model output using a training example; the walkthrough does not present independent evaluation with new questions.
Extra: CodeLlama Fine-tuning by code
The extra segment presents a code-based implementation for comparison with AI Quick Actions. The same JSONL dataset is converted to CSV, and separate training and chat test applications are used. The presenter reviews parameters and settings, starts training and speeds up that part of the video.
Test of code-trained chat
The extra chat application is run and queried. Its response includes some dataset information, but the narrator explains that further adjustments are needed for precision. The closing compares this implementation work with the OCI Data Science AI Quick Actions preparation.