Summary
How can data governance connect architecture decisions with business goals?
The talk applies DAMA-DMBOK concepts to a hypothetical case involving document and structured-data ingestion and transformation in Oracle Cloud. It connects an architecture using OCI Object Storage, OCI Events, OCI Functions, and Oracle Autonomous Database with security, quality, metadata, and traceability.
Responsibilities, coordination between departments, and isolated sources are reviewed. An illustrative nine-month roadmap supports a discussion of migration, coexistence, standardization, early value delivery, and TCO and ROI assessment, including operational effort.
The walkthrough introduces Database Actions and Data Studio, a comparison of cloud platforms, and examples involving embeddings, medallion architecture, and workflows. It ends with questions about process dependencies and learning to work as a data architect.
Start
Joel Gangini introduces the talk and its relationship to data management in Oracle Cloud. The goal is to connect business needs with an architecture that prepares information for analytics and artificial intelligence.
Practical case: ingestion and data value
A case involving data ingestion, transformation, and administration is introduced. The starting question is how to obtain value from information and develop scalable projects, using DAMA as a reference.
Project challenges, goals, and returns
The hypothetical case combines documents and on-premises databases, batch processing, and real-time needs. The proposal is to define challenges, goals, and indicators that explain the investment and help assess improvements in time and cost.
Data architecture and event-driven ingestion
The proposed architecture stores files in a bucket and triggers events that invoke processing functions. Depending on the content, the workflow can include transcription or OCR before centralizing the information for consumption.
Autonomous Database and converged data
Autonomous Database is presented as a repository for relational data, JSON, and vectors. The architecture connects this information to AI models and applications and includes an execution layer to serve users.
Adoption phases and the DAMA framework
The adoption phases are explained as depending on the organization and the teams carrying out the project. DAMA is introduced as a reference for organizing responsibilities and capabilities, regardless of the chosen infrastructure.
Data management pyramid and AI consumption
The walkthrough covers modeling, storage, security, integration, and analytical consumption layers. The speaker adds AI and a color classification to his adaptation of the pyramid, distinguishing these personal contributions from the book’s content.
Governance and coordination between departments
The talk explains why the starting point depends on the project’s needs. An example of inconsistent student lists illustrates how poor communication, isolated sources, and duplicated work make reliable information sharing difficult.
Data quality, metadata, and traceability
Business rules and quality controls determine which information reaches consumers. Metadata is highlighted as a way to understand processes, reconstruct their history, and investigate incidents, alongside the tracking provided by logs.
Phased adoption proposal
The proposed approach starts by exploring existing integrations and processes, then develops the design while allowing services to coexist during the transition. Standardization follows an assessment of which components add value to the case.
Architecture, governance, and development roadmap
The pyramid’s layers are linked to workstreams within the roadmap. The reference architecture changes as needs, constraints, and sizing requirements are discovered; design continues throughout the project.
Roles and data ownership
The proposal identifies data owners from the start and coordinates teams around priority use cases. Governance requires people with decision-making authority who take responsibility for information in their domain.
Early value delivery and indicators
The proposal delivers indicators and results during the project, according to its scope, instead of waiting until completion. The talk discusses how integration, analytics, and AI tools can support delivery within the illustrative nine-month roadmap.
Migration, coexistence, and standardization
Three stages distinguish what to move, retain, and retire. Coexistence allows results, times, and costs to be compared between old and new components before standardizing the selected architecture.
TCO, ROI, and operational costs
The economic assessment includes infrastructure, development, and people’s effort on operational tasks. The proposal identifies this work and its hourly cost to assess automation, without attributing guaranteed savings or returns to the case.
The DAMA wheel and its domains
The DAMA wheel is reviewed with data governance at the center and domains such as quality and security around it. The speaker explains that project needs guide the entry point and the path through the chapters.
Database Actions and Autonomous capabilities
The walkthrough covers Database Actions and Data Studio categories: development, administration, monitoring, and data management. Modeling, JSON, graph, and catalog utilities are mentioned as capabilities to explore when designing the solution.
Aligning services with DAMA domains
The proposal relates each service’s capabilities to specific needs in the DAMA domains. The solution combines complementary tools; the comparison helps plan future capabilities and does not constitute a compliance certification.
Comparing services and multicloud architectures
Storage services from AWS, Azure, Google Cloud, and Oracle Cloud are compared to help explain a hybrid architecture. The talk proposes evaluating which components to combine according to the environment and project needs.
Embeddings architecture with Events and Functions
The talk returns to files in a bucket, events, and functions to explain an embeddings-processing architecture. It discusses how code connects the stages to the repositories that AI applications will use.
Medallion architecture and hybrid integration example
A clip from another video explains medallion layers and coexistence between on-premises infrastructure and cloud services. It highlights labeling each service with its function to make data movement and processing understandable.
Data framework: staging, catalogs, and control
The walkthrough reviews staging, catalogs, rules, processing frequency, and volume. The example includes control, monitoring, parameters, metadata, and mappings; it proposes that each workflow produce one dataset to organize its dependencies on other workflows.
Questions: process orchestration and dependencies
A question asks how to run a process that depends on others scheduled at different times. The answer discusses Airflow, PowerCenter, Oracle Data Integrator, and cloud options, relating the choice to available tools and environmental dependencies.
Learning path toward data architecture
The speaker shares his path from development to data architecture. He highlights problem-solving, ongoing research, and knowledge of a programming language, mentioning Python and PySpark as part of his suggested learning path.
Resources related
- DAMA International — DAMA-DMBOK and data management
- Oracle Database Actions — Development, data, and administration tools
- Data Studio — Catalog and metadata in Autonomous AI Database
- OCI Events — Event-driven automation
- OCI Functions — Serverless processing and event integration
- Joel Gangini projects on GitHub