By Siddartha Reddy Katukuri, HEXstream data integration engineer\nYour ERP imports are running…your audit trail is not. \nEvery utility has integrations running. What most don't have is proof of what happened between the file leaving the source system and the data appearing  or not appearing inside Oracle Fusion ERP.\nUtilities run on a mesh of platforms: Time and attendance, Storm tools, Fleet systems, Payroll vendors, Procurement, Billing. \nIn 2026, bulk data still moves as files. That's not old school. It's the most reliable way to move high-volume vendor data into Oracle Fusion ERP: GL journals, AP invoices, Supplier updates, Assets, Project costs, Labor allocations, and Expense feeds.\nThe real issue isn't file-based imports. It's this: When an ERP import fails or duplicates, most teams can't prove what happened between SFTP and ERP. Then we usually witness the scramble:\nSearching Fusion Scheduled Processes for import history\nHunting through email threads and SFTP timestamps\nChecking disconnected logs across multiple systems\nRe-running jobs "just in case"\nArguing about whether the file was sent, received, or processed\nThe cost isn't just hours...it's confidence. Consider delayed payments, missed reporting timelines, audit headaches, and staff time burned on work that should never repeat.\nThat model works when volume is low, the team is experienced, and nothing goes wrong at the same time. Close season and storm events are precisely when none of those conditions hold.\nWith structured OIC logging, that changes: every import gets a tracked outcome within minutes. Finance knows what passed, what failed, and what rejected, before anyone has to ask.\nWhat OIC File Server logging actually means\nOIC File Server is a managed, SFTP-compatible repository inside Oracle Integration Cloud,  the controlled staging layer where source systems drop files and OIC picks them up to drive ERP import pipelines.\nLogging isn't "more text in logs." It's an audit trail that answers what business teams actually care about:\nDid the file arrive?\nDid OIC process it?\nDid ERP import it?\nWhat was the outcome: success, rejects, or failure and why?\nWe are talking about governance, visibility and accountability delivered to the people who need to act on it.\nThe proven pipeline\nEvery well-governed utility OIC implementation follows the same path:\nSFTP → OIC File Server → Validate → Fusion ERP (FBDI/ESS) → Reject Handling → Notification\nMost teams build the pipeline. Most skip the part that makes it observable. That's the gap this approach closes.\nThe one design decision that makes everything else work\nBatch ID as a first-class control. Batch ID is how business and IT speak the same language about the same file.\nWithout a Batch ID, "find the file" is a cross-system search touching SFTP logs, OIC activity streams, Fusion Scheduled Processes, and email chains. With a Batch ID, one query surfaces the complete story, from SFTP arrival to ERP outcome in seconds.\nBatch ID must appear in every layer:\nFile name\nFile header row\nOIC tracking fields and business identifiers\nESS job reference and comment field\nReject file names\nNotifications\nNaming standard examples:\nTIMESHEETS_2026-02-25_BATCH_0098.csv \nPAYROLL_2026-02-25_BATCH_0142.csv \nSTORMWORK_2026-02-25_REV03_BATCH_2201.csv\nWhy the REV suffix matters: storm work and operational files get corrected and resent routinely. A revision marker signals a controlled update, not an accidental duplicate. That single detail is often the difference between a governed reprocess and a duplicate posting.\nSo how doesBatch ID outperform every other tracking approach? There are a number of ways: \nSearching by date returns every file that ran that day\nSearching by module returns every GL file ever processed\nSearching by checksum requires technical access most business users don't have\nSearching by Batch ID returns exactly one file, one run, and its complete history—and every person involved already has it from the notification or reject report in their inbox\nThe 5-minute troubleshooting runbook\nWe should measure outcomes, not noise. Consider: \n% of files completing within SLA: Your primary health signal. A drop here needs attention before it becomes a close incident.\nBacklog beyond expected processing time: Files sitting past their expected dwell time are waiting to become incidents. Catch them before finance does.\nPeak readiness: Storms and month-end are predictable events. Throughput capacity should be confirmed against both before they arrive, not during.\nWhen someone says "the file didn't load," start here. Not in Fusion. Not with the source system team.\nLet's break this down into steps. \nStep 1: Search by Batch ID in OIC tracking. One search. Complete lifecycle. Resolves the majority of inquiries without escalation.\nStep 2: Identify where it stopped. File Server, validation, or ERP job. Each location points to a different fix.\nStep 3: If it reached ERP, capture the ESS job outcome and reject details. The ESS output file contains structured row-level rejection reasons ready to act on.\nStep 4: Communicate rejection reasons in plain language with record counts. Reasons and counts. Not stack traces.\nStep 5: If backlog grows, throttle and requeue deliberately. Let critical feeds clear first. Controlled requeue order prevents the backlog from compounding during peak periods.\nThis is the operational maturity step that eliminates war rooms during close and storm events.\nWhy this is a business-controls decision\nWhen OIC File Server logging is built around Batch ID, it delivers many outcomes that truly matter. \nFaster close with fewer surprises: Finance knows every import status before reconciling starts.\nFewer duplicates and less rework: Revision markers and governed identifiers catch resent files before they become duplicate postings.\nResilience during storms and month-end peaks: Every file is identifiable and manageable by Batch ID regardless of volume.\nOne root cause shows up everywhere: data moves without visibility and accountability.\nOne fix: Treat Batch ID as the control that makes file-based ERP imports governable.\nOf course, the next close cycle starts whether this is in place or not. The only variable is whether your team spends it reconciling or scrambling.\n\nCLICK HERE TO CONNECT WITH US ABOUT OPTIMIZING YOUR FUSION PROGRAMS. \n
\n

Skip The Scramble: Tips On OIC File-Server Logging For Oracle Fusion ERP Imports

By Ramaswamy Iyappan, HEXstream data engineer\nAgentic AI is quickly moving from “chat with my data” demos to mission‑critical systems that analyze revenue, detect risk, and trigger real‑world actions. As that shift happens, one thing becomes painfully clear: LLMs alone are not enough. You can bolt tools and vector stores onto a model, but without a semantic model—a shared, explicit representation of your business concepts—agentic systems become brittle, hard to govern, and impossible to scale safely across teams and use cases. \nIn this post, we’ll cover:\n•	What a semantic model is in the context of agentic applications\n•	Why it’s necessary (and what breaks without it)\n•	How we’ve implemented it in GenBI at HEXstream at a high level\nWhat is a semantic model in an agentic system?\nIn BI, a semantic model usually means a curated layer of metrics, dimensions, and entities that hide raw schema complexity. In agentic systems, you can think of the semantic model as:> A machine- and human-readable map of your business concepts, relationships, and computations that every agent is required to use when reasoning about your data. Concretely, it typically includes:\n•	Business entities: Customer, Order, Product, Subscription, etc.\n•	Metrics: total_revenue, active_users, churn_rate, average_order_value, defined with precise formulas.\n•	Dimensions: by_region, by_plan, by_channel, by_time_grain, etc.\n•	Relationships & constraints: how tables join, valid filters, allowed time grains, role‑based access, etc.\n•	Canonical naming: mapping natural language phrases (“paying customers”, “trial users”, “MRR”) to stable, governed concepts.\nIn a modern agentic app, the semantic model becomes a central contract between:\n•	The LLM/agent (which interprets natural language),\n•	The data layer (SQL, warehouses, APIs),\n•	And governance (who can see what, how metrics are computed, and which actions are allowed).\nWhy semantic models are necessary for agentic apps?\n1. Reliability: From “best guess” to governed answers\nWithout a semantic model, an agent translating “Show me churn by region for last quarter” must:\n•	Guess which tables to use\n•	Guess how to compute “churn”\n•	Guess which date field represents “last quarter”\nThat might work in a sandbox, but in production you get:\n•	Inconsistent metrics across teams\n•	Subtle SQL errors (wrong joins, filtered out edge cases)\n•	Non‑reproducible answers when prompts or models change\nWith a semantic model, the agent instead:\n•	Resolves “churn” → churn_rate metric with a defined formula\n•	Resolves “by region” → region dimension on Customer\n•	Resolves “last quarter” → a standardized time grain / calendar logic\nThe model no longer “invents” business logic; it selects from a governed catalog.\n2. Safety & governance: Guardrails for powerful agents\nAgentic systems can:\n•	Trigger workflows (e.g., send emails, create tickets, adjust budgets)\n•	Combine data across sensitive domains\nWithout a semantic layer, you’re relying on ad‑hoc prompt instructions to enforce rules like:\n•	“Never show raw PII to non-admins”\n•	“Don’t join warehouse X with warehouse Y”\n•	“Only use this experimental metric in sandbox environments”\nA semantic model lets you encode these as first-class constraints, so agents:\n•	See only permitted entities and metrics given the user’s role\n•	Are restricted to allowed joins, filters, and actions\n•	Can be audited by reviewing semantic‑level plans, not opaque LLM decisions\n3. Maintainability: One place to evolve your business logic\nAs your business changes, business logic must change too:\n•	New pricing plans, new product lines\n•	Revised definitions (e.g., changing how you define “active user”)\n•	New data sources and warehouses\nIf each agent or prompt encodes its own logic, every change becomes an N‑way migration problem. With a semantic model:\n•	You update the metric definition once (e.g., active_users)\n•	All agents and experiences automatically align with the new logic\n•	Documentation, lineage, and tests live alongside the semantic model\n4. Better UX: Natural language that “really understands” your business\nUsers don’t think in tables; they think in concepts:\n•	“Enterprise customers on the new usage‑based plan”\n•	“Net retention for self‑serve customers in EMEA”\n•	“SQL errors for the last 24h in our analytics pipeline”\nA semantic model lets the agent:\n•	Map ambiguous phrases to canonical concepts\n•	Ask clarifying follow‑ups based on known entities and dimensions\n•	Provide explanations and drill‑downs in business language, not schema jargon\nThe result is a system that feels like it actually understands your business, not just your database.\nHow we use a semantic model in GenBI at HEXstream\nGenBI is our agentic BI application at HEXstream: a conversational interface that lets users explore metrics, slice and dice data, and reason about performance across domains—all backed by a robust semantic layer. Here’s a high-level view of how we’ve built this.\n1. A dedicated semantic layer describing the business\nWe maintain a semantic-layer module that describes:\n•	Entities like Account, Workspace, Run, Event\n•	Metrics such as total_runs, successful_runs, latency_p95, active_workspaces, revenue_mrr\n•	Dimensions including time grains (day, week, month), geography, plan type, environment, etc.\n•	Relationships: which entities can be joined, on which keys, and with what cardinality\nThis layer acts as a single source of truth for:\n•	Metric definitions (aggregation, filters, windowing rules)\n•	Default filters (e.g., how we define “active” or “current period”)\n•	Role- and tenant‑level scoping rules\nThe LLM never directly “guesses” which tables to use; it instead targets semantic entities and metrics, which are then compiled to concrete queries.\n2. Semantic planning before query generation\nWhen a user asks a question in GenBI, our agent doesn’t jump straight from text to SQL. Instead, it passes through a semantic planning step:\nA—Intent & entity detection\n•	Extracts what the user cares about (e.g., “churn”, “enterprise customers”, “last two quarters”).\n•	Resolves these to metric IDs and dimension IDs in the semantic model.\nB—Plan construction\n•	Chooses the main entity (Customer vs Workspace vs Run).\n•	Attaches metrics (churn_rate, active_customers) and dimensions (segment, region, time).\nC—Validation against the semantic model\n•	Checks that the requested metric/dimension combination is allowed and meaningful.\n•	Applies type and join constraints (e.g., no cross-warehouse join that violates policy).\n•	Applies role-based restrictions to hide or adjust metrics for certain users.\nOnly once a semantic plan is validated do we move to physical query planning (SQL / engine‑specific queries). This significantly reduces broken queries and “nonsense” combinations.\n3. Agents that are “semantic-first”, not “schema-first”\nIn GenBI, agents don’t learn your warehouse schema directly. Instead:\n•	They are given the semantic model as their world: entities, metrics, dimensions, relationships, usage examples.\n•	Tool interfaces are defined at the semantic level:\no	e.g., run_metric_query(metric_id, dimensions, filters, time_grain) instead of “run arbitrary SQL”.\n•	The agents’ internal prompts stress: always reason using the semantic model, never invent metrics or columns that don’t exist.\nThis gives us:\n•	Better control: We can see exactly which metrics and dimensions an agent used.\n•	Easier debugging: When something looks odd, we inspect the semantic plan, not a random SQL blob.\n•	Fewer jailbreak-style failures: The agent can’t “decide” to bypass governance because its tools only expose governed abstractions.\n4. Reusability across experiences\nBecause everything flows through the same semantic layer, we reuse it across:\n•	The chat interface (free-form questions)\n•	Guided analysis flows (predefined question templates)\n•	Dashboards and visualizations (drag-and-drop powered by the same metrics/dimensions)\n•	Potential future agents (alerting, anomaly investigation, what‑if simulation)\nWhen we introduce a new metric—say, time_to_resolution for incidents—we:\n•	Add it once to the semantic model\n•	Immediately unlock it in chat, dashboards, and any downstream agents\n•	Inherit all the same governance and visibility rules automatically\nBenefits we’ve seen in GenBI\nBy committing to a semantic model at the core of GenBI, we’ve seen several concrete benefits:\n•	Drastically fewer broken or incorrect queries\no	Most “this looks wrong” bugs now trace back to the semantic definition (which we can fix globally), not ad‑hoc agent behavior.\n•	Faster onboarding for new use cases\no	New domains plug into GenBI by adding entities/metrics to the semantic layer, not by rewriting agent logic from scratch.\n•	Clearer governance & auditability\no	We can answer: “Which metrics were used in this decision?”, “Who can see this metric?”, “What does this metric mean?”\n•	Better user trust\no	Because we can show metric definitions and lineage, users quickly move from “LLMs are magic” to “this is a governed data product I can rely on.”\nClosing thoughts\nModern agentic applications need more than a great LLM and a few tools. They need a semantic backbone—a model of your business concepts that is explicit, governed, and shared across agents, queries, and experiences. In GenBI at HEXtream, we’ve made the semantic model the centerpiece:\n•	Agents reason in terms of entities and metrics, not tables and columns.\n•	Every query passes through a semantic planning and validation step.\n•	Governance, documentation, and evolution happen once, in the semantic layer, and propagate everywhere.\nIf you’re building agentic systems on top of real business data, the most impactful architectural decision you can make is to treat your semantic model as a first-class product, not an afterthought.\n\nCLICK HERE TO CONTACT US ABOUT REAPING THE BENEFITS OF GENBI AT YOUR ENTERPRISE. \n\n

Why Semantic Models Are Essential For Modern Agentic Applications

By John Apostolo, HEXstream full-stack data analyst\nChange data capture (CDC) is a way to capture row-level changes—inserts, updates, deletes—from a source system and move them downstream continuously or in near-real-time. Batch processing, by contrast, moves snapshots or time-bound chunks of data on a schedule (hourly, nightly, weekly), producing new outputs per run.\nLet’s explore…\nDesign principles for CDC\nCDC systems are built around freshness and fidelity. The biggest design choice is where you capture change: log-based CDC usually preserves the most detail with the least impact on the source, while triggers or application events can be organized more simply but risk gaps or performance overhead. \nCDC also forces you to pick your “truth model”—do you emit raw row changes, business events, or full-state snapshots? \nRow changes are great for auditability and replay, but they push complexity onto consumers, who must handle deduping, ordering and idempotent upserts. Because failures are inevitable, CDC designs should treat replay as normal: stable keys, clear sequencing, and a defined bootstrap/backfill path are not “nice-to-haves”…they’re the difference between a pipeline you trust and one you babysit.\nDesign principles for batch\nBatch optimizes for throughput and repeatability. The core tradeoff is freshness versus simplicity and cost. Full refreshes are the most reliable but can be expensive; incremental batch reduces cost but introduces watermark logic, late-arriving data handling, and partition repair policies. Batch architectures shine when you need heavy joins, large aggregations, and reproducible “as-of” reporting, where deterministic reruns matter more than minute-by-minute updates.\nCDC-optimized databases\nDownstream stores for CDC should be comfortable with constant upserts and deletes. That often means minimizing indexes, partitioning for write locality, and choosing merge-on-read versus merge-on-write (query-time cost vs. ingest-time cost). Compaction and retention policies matter because tombstones and high churn can quietly degrade performance.\nBatch-optimized databases\nLastly, batch-optimized databases reward append-heavy loads, columnar storage, compression, and partition pruning. They prefer bulk inserts and partition overwrites over row updates, and they benefit from clustering/sort keys for common query patterns. The design bias is clear: scan fast, aggregate cheaply, and keep runs reproducible—even if “real-time” isn’t the goal.\n\nCLICK HERE TO CONTACT US TO OPTIMIZE YOUR PROCESSING STRATEGIES. \n
\n

CDC Vs. Batch Processing

By Venkatesh Bommakanti, HEXstream solutions engineering manager\nSecurity is a critical component of any cloud-based ERP system, and Oracle Fusion Cloud provides a robust, scalable, and enterprise-ready security framework. At the heart of this framework lies Role-Based Access Control (RBAC), which is implemented through two key components: Functional Security and Data Security.\nWhile Functional Security controls what actions a user can perform in the system, Data Security controls what data a user can see or access. Understanding the difference between these two is critical for designing secure, compliant and efficient access in Oracle Fusion.\nUnderstanding the distinction between Functional Security and Data Security is essential for designing effective access controls, preventing unauthorized access, and ensuring compliance with audit and governance requirements. This blog explores both concepts in detail, highlights their differences, and explains how they work together to create a secure and efficient security framework in Oracle Fusion.\n
\nUnderstanding Functional Security in Oracle Fusion\nFunctional Security in Oracle Fusion determines what actions a user can perform within the system. It controls access to application features such as pages, menus, tasks, buttons and business transactions. In simple terms, Functional Security answers the question, “What can the user do in the system?”\nHierarchical Functional Security Model\n\n\n\nFor example, an accounts-payable clerk may have functional access to create and view invoices, while an accounts-payable manager may additionally have access to approve invoices and manage payments. If a user cannot see a menu, button or page, it is typically a Functional Security issue rather than a data issue.\nBy aligning system privileges with job responsibilities, Functional Security helps organizations maintain a secure, controlled and efficient operating environment in Oracle Fusion Cloud.\nUnderstanding Data Security in Oracle Fusion\nData Security in Oracle Fusion determines what data a user can access, view or manage within the system. Unlike Functional Security, which controls actions and menus, Data Security controls access to specific business records such as invoices, purchase orders, projects, ledgers, suppliers, customers or inventory transactions. In simple terms, Data Security answers the question, “What data can the user see?”\nOracle Fusion implements Data Security through Data Security Policies, Security Profiles, and Data Access Sets, which restrict access based on business attributes such as business unit, ledger, legal entity, project, inventory organization or supplier. These policies are associated with roles, ensuring that users see only the data relevant to their responsibilities.\nFor example, an accounts-payable clerk in India may have access to view and process invoices only for the India business unit, while a US-based AP manager can access only US invoices. If a user can open a screen but not see any records, it is typically a Data Security issue, not a functional one.\nBy ensuring that users access only the data they are authorized to view, Data Security plays a crucial role in maintaining trust, compliance and data integrity within Oracle Fusion Cloud.\n
\nOracle Fusion: Functional Security vs. Data Security\n\n\n\nBest practices for Functional and Data Security in Oracle Fusion\n·     Clearly separate Functional and Data Security design—Always treat functional access (what users can do) and data access (what users can see) as two distinct layers of security. Designing them separately ensures better control, easier troubleshooting, and clearer governance over user access.\n·      Leverage seeded roles wherever possible—Oracle delivers well-designed, industry-standard seeded roles that align with typical job responsibilities. Using these roles minimizes customization effort, reduces risk, and ensures better compatibility with future updates and patches.\n·      Avoid modifying seeded roles; instead, create custom copies—Never change delivered Oracle roles directly. Instead, copy the relevant seeded role and make your changes in a custom role. This protects your configuration during upgrades and makes security maintenance more manageable.\n·      Follow the principle of least privilege—Grant users only the minimum level of access required to perform their job duties. This reduces security risks, prevents misuse of access, and supports compliance with audit and governance requirements.\n·      Conduct periodic reviews of data access—Regularly review user roles and data access to ensure they remain appropriate as job responsibilities, business structures, or organizational policies change. This helps prevent accumulation of unnecessary or excessive access over time.\n·      Validate all roles in UAT before deploying to production—Thoroughly test new or modified roles in a UAT or test environment before moving them to production. This helps identify issues early, avoids business disruptions, and ensures users receive the correct access from day one.\n\nCLICK HERE TO CONTACT US ABOUT ORACLE FUSION AT YOUR ENTERPRISE. \n


Functional Security Vs. Data Security In Oracle Fusion

By Gokuleshwar Raju Nannapuraju, data solutions engineer\nIn many utility organizations, analytics and reporting grew—over time—around existing operational systems. Reporting was spread across multiple legacy tools and processes such as RPD, OFS, rate-comparison reports, audit extracts, NACHA files, postal systems, activity statements, reconciliation outputs, and years of historical reporting logic. \nWhile these solutions worked at the time, they gradually became harder to maintain and increasingly manual, with teams wasting time and effort reconciling numbers.\nAs utilities move to Oracle Customer to Meter (C2M), reporting is also being revisited. C2M brings core customer, billing and meter data into a single system, which immediately reduces the complexity created by multiple upstream sources. This makes it easier to trust the data and removes much of the manual effort that was previously required to produce basic operational and regulatory reports.\nFrom an analytics standpoint, Oracle Utilities Analytics (OUA) provides a solid out-of-the-box starting point. OUA already includes standard metrics across areas such as credit and collections, customer analytics, meter-data analytics, and revenue analytics. For many utilities, this allows a large portion of legacy and custom reports to be retired early in the transformation.\nAt the same time, most utility reporting needs go beyond what is available out-of-the-box. As a result, organizations are extending OUAW to support additional dashboards and KPIs that reflect their specific business and regulatory requirements. Common examples include analytics related to credit activity, detailed account history, gas compliance, mass-billing validation, regulatory pricing, and economic development. These extensions enable utilities to keep a standardized analytics platform while still addressing local and regulatory needs.\nThe shift to modern analytics is less about introducing new reports and more about changing how reporting is used day-to-day. With C2M as the system of record and OUA supported by customized OUAW, reporting becomes more consistent and accessible. Teams spend less time validating data and maintaining legacy logic, and more time using analytics to support operational decisions, audits and regulatory discussions.\nOverall, simplifying the reporting landscape and aligning it with C2M creates a more sustainable analytics foundation. As legacy systems are retired, utilities gain better data reliability, clearer visibility into operations, and a platform that can evolve as business and regulatory requirements change.\n\nCLICK HERE TO CONTACT US ABOUT SIMPLIFYING YOUR REPORTING. \n\n\n
\n

Simplifying Utility Reporting: From Legacy Reports To Modern Utility Analytics

By Ashwini Nagendra Prasad, HEXstream solutions engineering manager\nUtility outages are inevitable. What defines performance isn’t \nif\n outages happen—it’s how effectively the organization responds.\nDuring a major outage, customers don’t care about OMS vs. CIS or AMI vs. ADMS or back-end system boundaries. They care about accurate information, fast restoration, and clear communication. That’s where integration architecture quietly becomes mission-critical.\nBusiness reality \nOutages are moments of truth. Call volumes spike. Customer trust is on the line. Operations teams need a single, reliable view of the grid. Leadership and regulators expect clarity and control.\nWe’ve seen that even with modern operational systems in place, utilities struggle—not because systems fail, but because they don’t work together in real time.\nThe integration challenge exposed by outages\nDuring outage events, common integration gaps surface quickly. Status updatescan  lag across systems. Batch and polling-based integrations call fall behind. Point-to-point dependencies can create cascading failures, all while different systems tell different stories. \nThe result is confusion, manual workarounds, and delayed decisions exactly when speed matters most.\nThe architecture shift\nOutage response isn’t a system problem. It’s a coordination problem.\nModern integration architectures enable:\nEvent-driven communication instead of polling\nReal-time propagation of outage and restoration events\nLoose coupling so systems scale and fail independently\nA shared operational truth across customer, field and control systems\nWhen an outage is declared or restored, every system should know at the same time—not minutes or hours later.\nThe architecture principle\nSmart uilities design integrations for event-driven collaboration, not system ownership. Consider how outages unfold as business events—the outage is detected, the crew is assigned, restoration times are estimated and, ultimately, power gets restored\nThe best integration strategies should reflect that reality—enabling systems to react, not wait. During outages, technology should reduce complexity—not add to it. Integration architecture determines whether systems collaborate seamlessly or operate in silos when it matters most.\nIn the utilities domain, integration isn’t middleware. It isn’t plumbing. It’s a reliability enabler. We at HEXstream believe that when the lights go out, integration architecture determines whether systems succeed by collaborating—or fail by competing.\n\nHow is your organization approaching integration for outage management and grid
operations? Click here to contact us about our new HEXaid platform.\n

When The Lights Go Out, Systems Must Work Together

By Bharath Moola, HEXstream data solutions engineer\nFor as long as most of us have been building software, there has been a massive divide in the development world.\nOn one side, you have the application developers. They live in the world of JSON because it is flexible, nested and exactly what the front end needs to render a page. On the other hand, you have database engineers. They trust relational tables because they want data to be normalized, indexed and transactionally bulletproof.\nProject kickoffs usually start with the same architectural tug of war: Do we go with a document store for speed of development, or relational tables for data integrity?\nUsually, we settle for a messy compromise. We use an Object Relational Mapper (ORM) that performs complex data transformations, or we build sync jobs that eventually break. However, it is now 2026, and we are finally moving past that struggle. Oracle JSON Relational Duality Views have turned that "either/or" choice into “both" solutions.\nThe pick-your poison era\nBefore this technology, we were stuck with three frustrating options:\n1.     Pure relational: This was great for data integrity but a nightmare for APIs. You would spend half your sprint writing complex joins just to get a simple nested object for a mobile app.\n2.     JSON in a blob: This was fast to build, but it was difficult to run a performance report or ensure a field was not missing. It was essentially a digital junk drawer.\n3.     Double life: This meant storing data in tables and syncing it to a separate document store. This led to the inevitable Friday night crisis where the API showed different data than the database because a sync job failed.\nEnter duality views: The live bridge\nThink of a Duality View as a bridge between two worlds. It allows you to store your data once, properly normalized in relational tables, while interacting with it as if it were a native JSON document. This is not a copy or a cache; it is a live contract.\nIf you update a value in the JSON document, the underlying rows in your SQL tables update instantly. If a database administrator runs a SQL script on the tables, the JSON view reflects those changes immediately. It is the first time both sides of the house are looking at the exact same truth in the format they prefer.\nA real-world example: Incident management\nLet us say you are managing network outages. In the traditional relational world, an event record requires joining tables for events, causes, and affected services.\nWith a Duality View, your development team simply sees a clean JSON object containing the event ID, the status, the start time, and a nested details object.\nThe most impressive part is that you can send that entire JSON object back to the database, and the system handles the heavy lifting of updating the correct rows across three different tables. No mapping code or manual data shredding is required. It simply works.\nWhy this is a game-changer\nThis approach removes the friction that has slowed down development for years.\nSkip the complex mapping layers: Your REST or GraphQL services can talk directly to the view. You can stop debugging lazy loading issues and focus on building features.\nAnalytics still win: Your business intelligence tools and SQL queries still see clean, indexed tables. You do not have to sacrifice reporting speed for developer agility.\nZero-sync lag: Because there is only one source of truth, there is no background process moving data and no risk of the API getting out of step with the database.\nBuilt-in safety: Duality Views use optimistic locking. If two people try to edit the same JSON document at the same time, the database handles the conflict.\nThe bottom line\nWe are finally moving away from an era where we had to choose between data formatted for computers and data formatted for humans.\nWith JSON Relational Duality, you get the flexibility of a document store combined with the industrial-grade power of a relational engine. It is about time we stopped compromising.\n\nWANT MORE? CLICK HERE TO CONTACT US. \n\n


The Great Data Divorce Is Over: Why We Are Finally Done Choosing Between JSON And Tables

By Praveen Reddy Atturi, HEXstream solutions engineering manager\nFirst, let’s define what we mean by PII attributes.\nPersonally Identifiable Information (PII) refers to any data that can identify an individual, such as their name, email address, social security number (SSN), salary or performance rating. In data security, it is crucial to ensure that PII attributes are protected and not visible to all users, even those within the same functional area. \nFor example, while multiple users may have access to the payroll module, only authorized users should be able to view compensation-related KPIs. This blog post outlines the detailed steps to secure sensitive data by hiding specific columns within a subject area using the security features available in Oracle Fusion Data Intelligence.\nIntroduction\nOracle FDI provides object-level security at both the subject area and workbook levels. However, the prebuilt object-level security does not account for restricting PII attributes, which are often included across multiple FDI subject areas. To ensure data privacy and compliance, these sensitive attributes can be restricted by implementing custom security in FDI, utilizing either prebuilt or custom application roles to control user access effectively.\nPrerequisites\nBefore implementing column-level security, ensure the following prerequisites are met:\nCreate or identify an application role that will be used to restrict access to PII attributes. In this example, the role of custom restricted PII access is used.\nAssign the user who should have restricted access to the PII attributes to a group role that is mapped to the custom restricted PII access application role.\nImplementation steps\nAccess semantic-model extensions from the FDI console, navigate to semantic model extensions.\nOpen security configurations under security configurations, select configure object permissions and click next. This displays a list of both prebuilt and custom subject areas available in the environment, along with their current security configurations.\nSelect the subject area and PII attribute In the left pane, navigate to the required subject area and expand the folders to locate the specific column to be restricted. For example, the PII attribute column under a specified table is selected for restriction.\nRestrict access for authenticated users after selecting the column, set authenticated users to no access on the right-hand side. This ensures that access to the column is controlled exclusively through role-based permissions defined in the semantic model.\nRestrict access for custom application role click show all roles, search for the application role identified in the prerequisites section (for example, custom restricted PII access) and set its access level to no access. Review your configuration and click finish to apply the changes.\nBefore vs. after implementation comparison\nBefore implementing the configuration, the #column was visible to all users with access to the corresponding subject area, including those who should not have permission to view sensitive information.\nAfter applying the custom security configuration, the #column attribute is successfully hidden from users who do not have the appropriate access privileges. This ensures that only authorized roles can view PII-related data while maintaining smooth functionality across the subject area and related workbooks.\nImplementation summary\nTry implementing this configuration in any subject area containing PII attributes that require restricted access. Once applied, the security rules automatically extend to all FDI workbooks using that subject area, ensuring that sensitive columns remain hidden from unauthorized users without additional setup.\n\nCLICK HERE TO CONTACT US ABOUT ENHANCING YOUR SECURITY STRATEGIES. \n\n\n
\n

Implementing PII Attribute Security In Oracle Fusion Data Intelligence

By Swathi Ambati, HEXstream solutions engineering manager\nFinancial data security isn’t just an IT concern; it’s a core finance and governance requirement. As organizations grow across legal entities, business units and regions, controlling who sees which financial data is essential for accuracy, compliance and accountability.\nOracle Fusion Data Intelligence (FDI) centralizes operational and financial data, enabling analytics while enforcing robust, enterprise-grade security. But fully understanding how this security works and when to customize it is critical for organizations with complex reporting needs. Lets explore...\nHow Oracle FDI security works\nFDI secures analytics in two ways:\n1.       Content access (controlled by application/duty roles) decides which dashboards or reports a user can open.\nData access (controlled by data roles and security assignments) decides which financial data the user can see in those reports.\nThese controls rely on application roles, data roles, security assignments and semantic-model mappings. Security rules are applied dynamically at runtime, ensuring users only see the data they’re authorized to access.\nExample: Alice is a finance manager for USA Corp. She opens the Profit & Loss dashboard and sees only USA Corp’s financial data. Europe data is automatically hidden.\nKey components of FDI security\nBelow, I’ve summarized the key components of FDI security. For a more detailed explanation of Oracle Fusion and FDI security concepts, you can also refer to my colleague’s \nblog\n. \n1. Application roles\nDefine who can access analytics and which subject areas they can use.\nPrebuilt roles exist, but custom roles are often required for unique business scenarios.\nExample: Finance manager can open General Ledger dashboards but may need additional data roles to restrict data visibility.\n2. Security contexts and assignments\nDetermine which data values users can access, such as ledgers, business units or balancing segments.\nExample: Assign Alice only Legal Entity = USA_Corp. Reports automatically filter to show only permitted rows.\n3. Semantic-model security configuration\nSemantic-model security mapping links role‑based security assignments to semantic-model attributes so that FDI automatically applies the corresponding data filters at query time, ensuring users only see the data they are authorized to access.\n4. Row-level data enforcement\nApplies security dynamically at runtime, ensuring unauthorized data is never visible.\nWhy custom security may be needed\nWhile prebuilt roles handle many scenarios, organizations often require fine-grained, custom controls such as:\nRestricting data by balancing segment or cost center\nLimiting visibility by business unit or hierarchy\nEnforcing compliance or audit-specific rules\nExample scenarios:\nAlice should only see US Corp., and within that only the East Division balancing segment.\nA business unit manager can see all cost centers within their unit.\nA CFO can view consolidated data across all legal entities.\nCustom security ensures that each user sees exactly the data they are authorized for, while maintaining centralized, consistent and audit-ready controls. Custom security uses configurable security contexts, allowing centralized, consistent and audit-ready controls while respecting organizational policies.\nBenefits of centralized and custom security\nNo report maintenance: Security updates automatically apply to all dashboards\nConsistent enforcement: One rule applies across dashboards, KPIs and exports\nScalable: Works across ledgers, legal entities, COA segments and hierarchies\nAudit-ready: Centralized enforcement supports governance and compliance\nFuture-proof: Easily extendable to other COA attributes like cost center or natural account\nExample in practice: Bob has access to the same dashboard as Alice but sees only Europe GmbH data, without creating separate dashboards or manual filters.\nConclusion\nOracle FDI’s security framework ensures enterprise-grade, COA-based data protection that is reliable, scalable and aligned with best practices by combining:\nApplication roles\nSecurity contexts and assignments\nSemantic-model extension\nRow-level enforcement\nOrganizations can implement precise financial-data security, whether standard or complex, ensuring that the right people see the right data at the right time (without constant dashboard maintenance).\nBalancing segments are just one example, but they demonstrate how COA-based security in FDI empowers finance teams with trustworthy, compliant and scalable analytics.\nData security is more critical than ever. Also critical is choosing the right approach for your custom security. We at HEXstream are ready to effectively design and implement your security.\n\nCLICK HERE TO CONTACT US. \n\n

A Smarter Approach to Financial Data Security in Oracle Fusion Data Intelligence

By Venkatesh Bommakanti, HEXstream solutions engineering manager\nOracle Fusion Applications are built on a robust, role-based security framework that ensures data protection, compliance and controlled access across enterprise systems. Fusion Security is built on a hierarchical model consisting of roles, privileges, data security policies and security profiles, all working together to enforce a secure and compliant environment.\nFusion Security also provides comprehensive auditing capabilities that track changes to user access, roles, configurations and transactions. Oracle Fusion Security is essential for designing scalable solutions, troubleshooting access issues, building secure extensions and ensuring proper segregation of duties across the system.\nIn this blog, we will explore the Fusion Security Model in a clear, technical and practical way.\nUnderstanding Oracle Fusion Security Architecture\nOracle Fusion Security Architecture is a comprehensive, multi-layered framework designed to protect enterprise data, enforce compliance, and manage user access across cloud applications. It is built on a Role-Based Access Control (RBAC) model, where every user's access is governed by carefully structured layers of roles, privileges and data-security policies. This architecture ensures that users see only the data they are permitted to access and can perform only the actions aligned with their job responsibilities.\nAt the core of the architecture is a hierarchy of role types:\nJob roles define functional responsibilities.\nDuty roles break down tasks into smaller capabilities.\nPrivileges specify the exact operations a user can perform.\nData roles determine which business units, ledgers, or departments the user can access.\nAbstract roles (such as employee or line manager) provide foundational access required by most users.\nFusion Security also employs data security policies and security profiles to control visibility of specific data objects—ensuring segregation of duties and minimizing data exposure risks.\nOverall, the Oracle Fusion Security Architecture provides a scalable, modular and highly controlled environment that supports secure application access, granular data protection and comprehensive audit readiness.\n
\nTypes of security roles in Oracle Fusion\nFusion security is structured using multiple role types that work together in a hierarchy. Understanding each type is key for designing secure and maintainable access structures.\n1.    Job roles\nJob roles in Oracle Fusion represent the business responsibilities assigned to a user within the organization. They define what a person is allowed to do in the system based on their real-world job function.\nFrom a technical perspective, a job role acts as a top-level container that includes:\n·       Duty roles (task-level responsibilities)\n·       Privileges (specific actions a user can perform)\n·       Data access policies (scope of data visibility when combined with data roles)\nExample—A “payables accountant” job role may include capabilities like:\n·       Managing supplier invoices\n·       Validating transactions\n·       Running the payables trial balance\n·       Accessing payment files\n2.    Duty roles\nDuty roles in Oracle Fusion represent the task-level responsibilities a user must perform within an application. They sit below job roles in the security hierarchy and break down a job function into smaller, manageable components. Duty roles provide granular access control, enabling Oracle Fusion to maintain a secure, modular and organized security structure.\nHere is some common duty roles used across modules:\n·     Manage invoices duty\n·       View supplier information duty\n·       Maintain employee information duty\n·       Submit general ledger journals duty\n·       Manage purchase orders duty\nEach of these duty roles bundles the privileges needed to perform the specific task.\n3.     Privileges\nPrivileges in Oracle Fusion represent the lowest and most granular level of security. They define the exact action a user is allowed to perform within the application, such as viewing, creating, editing, deleting or approving specific business objects. In the Fusion security hierarchy, privileges sit beneath duty roles and job roles.\nExamples:\n·      Create invoice privilege\n·      Edit journal-entry privilege\n·      View employee person record privilege\n4.    Data roles \nData roles in Oracle Fusion control which specific data a user can access. While job roles and duty roles determine the functional actions a user can perform, data roles determine the scope of data visible to them—such as business units, ledgers, legal entities, departments or employee groups.\nData roles ensure users see only the data relevant to their business context, and provide data-level security using security policies and security profiles. They restrict the visibility to specific BU, Ledger, LE or HCM objects.\nExamples\n·       AP specialist – business unit XYZ\n·       Procurement manager – business unit ABC\n·       HR specialist – department 123\n·       Payroll manager – country-specific data role\n5.    Abstract roles \nAbstract roles are broad, foundational roles assigned to large groups of users regardless of their job function. They provide general, non-job-specific privileges required by most users in the organization. Examples of abstract roles include:\n·       Employee\n·       Line manager\n·       Contingent worker\n·       All users\nHierarchical security model\n\n \n\n\nData access in Oracle Fusion\nData access in Oracle Fusion defines which specific records or business entities a user is allowed to see and work with inside the application. While job roles and duty roles determine what actions a user can perform, data access policies determine on which data those actions can be performed. This ensures a secure, controlled and compliant environment where users interact only with data relevant to their business responsibilities. \nData access is governed by three primary components:\n1. Data roles\nData roles restrict visibility to specific business contexts such as:\nBusiness units\nLedgers\nLegal entities\nDepartments\nCountries or payroll groups\nSpecific groups of employees (HCM)\nA user may have the functional privilege to “view invoices,” but will only see invoices for the business units defined by their assigned data role.\n2. Security profiles (mainly in HCM)\nSecurity profiles define the set of records a user can view or maintain.\nCommon security profiles:\nPerson security profile – controls which employees are visible\nOrganization security profile – limits access by department or business unit\nPosition or payroll security profiles – restricts payroll or HR data\nThese profiles attach to data roles or custom security configurations.\n3. Data-security policies\nData-security policies map privileges to specific data objects. They define rules such as:\n“The user is permitted to view invoices only for Business Unit ABC.”\n“HR specialist can access only employees in department = 100”\n“The buyer can manage purchase orders solely for legal entity XYZ.”\nThese policies ensure enforcement of segregation of duties (SoD) and prevent unauthorized data exposure.\nBest practices for Oracle Fusion Security Design\n1. Do not modify seeded roles\nAlways copy seeded roles and create custom versions to avoid upgrade issues and maintain clean security architecture.\n2. Follow least-privilege access\nGrant only the exact privileges required for a user’s job. Avoid unnecessary access that may lead to security risks or SoD violations.\n3. Separate functional access from data access\nJob/duty roles control functions; data roles and security profiles control what data the user sees. Both must be designed correctly.\n4. Assign roles through groups or provisioning rules\nUse groups or auto-provisioning instead of assigning roles directly to users for easier management and auditing.\n5. Maintain a security matrix and documentation\nDocument role-to-privilege mappings and data access rules for audits, troubleshooting and governance.\n6. Test role access thoroughly in lower environments\nValidate navigation, approvals, data visibility and reporting access before moving changes to production.\nConclusion\nOracle Fusion Security is a powerful and flexible framework that ensures users have the right access to the right data at the right time. By understanding the hierarchy of roles, privileges and data-access policies, consultants can design secure environments that support business processes effectively while maintaining compliance and audit readiness.\nA strong security model leads to:\nBetter user experience\nReduced operational risks\nEasier maintenance\n\nCLICK HERE TO CONTACT US ABOUT ORACLE FUSION SECURITY. \n

Oracle Fusion Security: Roles, Privileges And Data Access

\n\n
⁠
⁠\nBy Srivyshnavi Akarapu, HEXstream data analytics engineer\n\nIn this blog, we will see how to customize out of the box (OOTB) dimensions by adding characteristics in Oracle FDI. This customization helps adapt the standard data model to meet business requirements and improves analytics. 

\nKey Points to Consider:\n

→The new table must contain matching primary key data to join with the out-of-the-box dimension.

→ The primary key column name in the new table must be different from the primary key name in the existing dimension.\n
\nFlow of steps: \n

→ Add the new dimension by selecting Data Source from the connectors in FDI, then run the job to create the table with data.

→ In FDI platform, create a sandbox and modify the existing logical star of the required subject area using semantic model extension feature.

→ Modify out of the box subject area and apply changes.

→ After publishing, check the newly added changes are reflected in the existing folders.

→ Final step is to verify the characteristics data is available in OAC reports.\n\n Detailed steps\n
Replicate the required Characteristic table via Data Augmentation\nNavigate to Console → Data Configuration → Desired Data Source and select Data Augmentation\nCreate Augmentation.\n 
\n\n  Select the options below:\n\n\n \nSelect the attributes required.\n\n\n\nEnter the details below and run the job.\n\n\n\nAdding characteristics to existing logical star\nWe will first add the characteristic table first to "Fact - Case" logical star by navigating   to Sandbox → Perform Action → Edit Logical Star.\n       
 \n\n\n\nRight click on Dim – Account dimension and select Manage Extensions.\n\n\n\n\nSelect Manage Extension → Extend Dimension, to extend an existing dimension by joining new table. This adds the characteristics attributes to the existing dimension.\n       
\n\n\n\nSelect the schema OAX_USER schema, this contains all the data augmented replicated tables which can be chosen to extend any dimension.\nWe chose the Data Augmentation replicated table: DW_GBU3_X_CI_ACCT _CHAR, select the required columns using ACCT_ID as the primary key.\nWe will join the char table to the  Account dimension using this key.\n\n\n\n\n\nSelect Account ID from Dim-Account dimension, join it with the key from the char table.\n\n\n Modifying subject areas\n\n\n\n\n\n\n\n\n\n       
\n\n\n\nSelect Manage Extensions → Manage Factory Customizations.\n       
\n\n\nChoose the extension we added in logical Dim – Acct char dimension.\n       
\n\n\n\nAdd to Folders and choose Account folder.\n       
\n\n·       We see that attributes from the char table used for extensions are added to the account folder.\n\n\n   \n\n    \n·       For the changes made in the sandbox to be reflected, we must Apply, Merge and Publish the model.

1) Click on the Actions Menu and select Apply Changes option, status of this job can be monitored under Activity tab.\n       
\n\n\n      2) Once Apply Changes is successful Merge to Main Sandbox, status can be monitored under Activity tab.\n       
\n\n\n       
  3) Now Publish the model by selecting User Extensions → Main and publish.\n  \nVerification\nNavigate to DV URL and under Data → Datasets section →select subject area Case.\n    \n\n\n\n\n\n\n\n\n\n


Adding Characteristics to OOTB Dimensions In Oracle OUDI FDI

By Venkatesh Bommakanti, HEXstream solutions engineering manager\nOracle Fusion applications provide a rich set of reporting and analytics tools designed for different business needs—from real-time operational reports to large data extracts and advanced analytics.

Two of the most widely used Oracle Fusion reporting tools are OTBI (Oracle Transactional Business Intelligence) and BIP (BI Publisher). Both aim to extract, analyse, visualize and deliver business data from Oracle Fusion Cloud applications in a structured, meaningful, and actionable way.\nIn this blog, we will explore what OTBI and BIP reporting tools are, compare their capabilities, discuss their advantages and limitations, examine their performance impact, and clarify when each reporting tool should be used.\nWhat is OTBI?\nOTBI is a real-time, self-service reporting tool in Oracle Fusion Cloud that enables business users to create analyses, dashboards and visualizations using prebuilt subject areas without writing SQL. It retrieves data directly from Fusion transactional tables and is ideal for operational and day-to-day reporting.\nKey features include real-time reporting to pull data directly from live transactional tables. The security-aware reporting feature ensures that Fusion’s role-based security is automatically applied, so users can access only the data they are authorized to see, with row-level and data-level security managed seamlessly. \nOTBI enables users to create reports using drag-and-drop components with business-friendly terms such as employee, supplier, invoice and expense. OTBI is a real-time, ad-hoc reporting tool that enables users to quickly create reports without IT support, making it ideal for functional users and business analysts.\nVISUAL FLOW OF OTBI REPORTING PROCESS\n\n
\nAdvantages of OTBI\n·      OTBI supports tables, pivot views, charts, KPIs, and drill-down dashboards for better analytics.\n·      It offers instant visibility into approvals, daily transactions, and business activities without waiting for data refresh.\n·      Automatic security ensures users only see data they are authorized for, minimizing risk.\n·      OTBI enables users to analyze trends, monitor performance, and identify issues instantly from a single view.\nDisadvantages of OTBI\n·       OTBI is transactional and real-time focused, so it is not suitable for historical and large data analysis.\n·       Heavy joins, large data sets, or complex formulas may slow down execution, leading to timeouts.\n·       Subject areas are predefined. Deep customization or new subject areas require BI Publisher, Data Warehouse or custom extensions.\nWhat is BIP? \nBIP is Oracle’s enterprise-reporting solution designed to generate formatted, pixel-perfect reports using data extracted from Oracle Fusion applications and other data sources. It is ideal for documents that need precise layouts such as checks, invoices and forms.\nBI Publisher reports can pull data from SQL queries, web services, XML files, OTBI extracts, Fusion tables, and even external databases. Reports can also be scheduled, emailed, printed or automatically distributed to multiple recipients based on defined conditions.\nBIP reports are designed to efficiently handle large datasets and support operational, real-time reporting. They also ensure compliance with statutory formats. They meet structured output requirements.
\nVISUAL FLOW DIAGRAM FOR BIP\n\n\n\n\n\n\n\n\n\n
\nAdvantages of BI Publisher\n·       Handles heavy datasets and complex reports with better performance compared to OTBI.\n·       Ensures statutory formats and structured document output, suitable for tax or legal reports.\n·       Supports sub-templates and data models for quicker development and consistency.\nDisadvantages of BI Publisher\n·       Building data models often involves SQL, joins, and XML knowledge, making it less friendly for non-technical users.\n·       Complex templates, multi-format output, and customization may increase development time.\n·       Users cannot drag-and-drop or explore data as with OTBI; reports are mostly predefined and static.\nOracle Fusion: OTBI vs BIP
\nPerformance is one of the biggest differences between the two schemas:\n\n
\nWhen to use which reporting tool in Oracle Fusion
\nUse the OTBI reporting tool when:\n·        You need real-time insights for operational dashboards or KPIs.\n·        Business users require self-service reporting without SQL.\n·        Interactive visuals like charts, graphs and pivots are needed.\n·        Role-based security must apply automatically.\n·        The required data exists within OTBI subject areas.\nUse the BIP reporting tool when:\n·        You need formatted, printable documents like invoices, POs or pay slips.\n·        Large data-extraction or bulk-reporting is required.\n·        Report-bursting is needed to send personalized outputs to multiple users.\n·        The report relies on SQL or data not available in OTBI subject areas.\n·        Scheduled or batch reports must run automatically through ESS jobs.\n
\nConclusion
\nBoth OTBI and BI Publisher play an important role in Oracle Fusion reporting—OTBI is best suited for fast, real-time, self-service analytics, while BIP excels in heavy data extracts, formatted outputs, and scheduled batch reporting. Selecting the right tool for the right requirement not only improves system performance and reduces maintenance efforts, but also enhances user experience and reporting efficiency across the organization.

Oracle Fusion OTBI vs BI Publisher

By Siddartha Reddy Katukuri, HEXstream data integration engineer\nThe utility industry is rapidly shifting toward cleaner, smarter and more connected energy ecosystems. With rising demand across electricity, gas, water, and renewable sectors, utility organizations must track everything from generation and consumption to asset performance and customer engagement—often in real time. Achieving this level of insight
requires robust, secure, and scalable integration across legacy platforms, cloud applications, field systems, and emerging digital services.\nThis is where Oracle Integration Cloud (OIC) running on Oracle Cloud Infrastructure (OCI) becomes a strategic differentiator. OIC helps utilities unify operations, automate business processes, and accelerate digital transformation by serving as a modern integration layer that connects systems end-to-end with speed, governance, and resilience.\nHow OIC powers next-gen integrations on OCI\nMost enterprises and especially utilities do not operate on a single application. Instead, they rely on a mix of ERP, CRM, CIS, OMS, DMS, GIS, MDM, WMS, custom apps, and legacy on-premise platforms. These systems must exchange data reliably and consistently to support customer services, grid operations, asset lifecycle management, and regulatory reporting.\nOracle Integration Cloud (OIC) provides the “Enterprise Integration Layer” that brings these ecosystems together. Built for hybrid and multi-application environments, OIC enables utilities to modernize integration without disrupting the core systems that keep the lights on.\nAI-driven insights and intelligent automation\nOracle Integration Cloud now embeds AI and machine learning capabilities to make integrations smarter and more adaptive.\n·      AI-assisted mapping: OIC automatically suggests data mappings between source and target schemas using pattern recognition, significantly reducing manual effort.\n·      Predictive monitoring: Built-in AI models detect anomalies and integration bottlenecks, allowing proactive resolution before failures occur.\n·      Process intelligence: OIC’s Integration Insight leverages AI to analyze process performance, recommend optimizations, and identify repetitive manual tasks for automation.\n·      Generative integration design: Oracle’s new AI Assist features generate integration flows and expressions from natural-language prompts—accelerating build time.\n·      Adaptive error handling: Intelligent fault management learns from past errors to suggest automated recovery or retry strategies.\nCore integration patterns supported by OIC\nOIC supports multiple integration approaches commonly required in large-scale integration programs:\n·      Support secure, API-based integrations to enable real-time data exchange and modular connectivity across systems.\n·     Support event-based synchronization to push updates instantly between systems.\n·      Provide flexible data transformation and schema mapping across applications (e.g., Dynamics Customer → CIS Customer, or Order → Work Order).\n·      Enable end-to-end process automation across CRM, ERP, and external systems with error handling, conditional routing, and approvals.\n·      Support hybrid integration by securely connecting cloud and on-premise enterprise environments.\n·      Offer centralized dashboards and monitoring to track integration performance and handle failures or retries gracefully.\n·      Deliver a cloud-native, scalable, and extensible platform suitable for high-volume transactional integrations.\nHigh-value utility use cases for OIC on OCI\nOIC delivers measurable impact across key utility systems such as WMS, CIS, OMS, DMS, GIS, MDM, ERP, CRM, and more.\n·      Workorder lifecycle orchestration: It handles real-time data exchange, status updates, and error handling, ensuring all operations and back-office systems remain perfectly aligned.\n·      Outage and restoration synchronization: When outages are detected, OIC synchronizes affected customer data, triggers work orders in WMS, and provides automated notifications to customers—improving reliability and response time.\n·       Asset and spatial-data integration: This ensures that when new assets are installed, replaced, or relocated, updates flow seamlessly across all systems—eliminating manual reconciliation.\n·      Customer and billing-data consistency: OIC ensures customer and billing information remains synchronized between CIS, CRM, and ERP platforms.\n·       Meter-data flow and validation: It validates incoming readings, handles data exceptions, and automates correction workflows — ensuring billing accuracy and compliance.\n·       Event-driven grid operations: OIC supports event-driven integration between DMS, OMS, and WMS, ensuring operational continuity.\n·      Data analytics and historical reporting: This enables enterprise-wide reporting, KPI tracking, and trend analysis across operational and customer systems—without impacting production workloads.\nWhy OIC should be central to your OCI integration strategy\nUtilities are evolving into highly digital, data-driven enterprises. Success depends not only on modern applications but on the integration backbone that connects them. Oracle Integration Cloud (OIC) on OCI offers a robust, future-ready platform to unify utility ecosystems, accelerate automation, and improve both operational reliability and customer experience.\nAs utilities invest in grid modernization, customer transformation, and renewable integration, OIC is well-positioned to serve as the integration engine that powers the next generation of resilient, intelligent utility operations.

Integrating business applications with Oracle Integration Cloud (OIC) on OCI

By Naveen Madishetti, HEXstream data analyst\nData modeling is the backbone of every successful Power BI solution. Whether building an executive dashboard or a complex analytics model, the structure of the data determines performance, flexibility and accuracy of the reports. 

Two of the most widely used data-modeling techniques in Power BI are the star schema and snowflake schema. While both aim to organize data efficiently, they differ in design, purpose and performance. In this blog, we will explore what star and snowflake schemas are, their advantages and disadvantages, how they impact performance, and when to use each schema.
\nWhat is a star schema?\nA star schema is a data-modeling technique where a central fact table connects directly to multiple dimension tables. The design is called a “star” because the fact table sits at the center, while the dimension table branches out around it, forming a star-like shape.

In the Power BI star-schema model, the tables are typically connected using a one-to-many relationship where each dimension table is on the “one” side and the fact table is on the  “many” side, ensuring clean, efficient, high-performance data modeling.\n\n\n                                                     \nStar schema\n

The fact table contains numerical data such as sales amounts, quantities sold and revenues. Along with these metrics, the fact table also includes foreign key columns that connect to different dimension tables. The dimension table has descriptive columns such as product details, customer information and dates.

\nAdvantages of star schema\n·       Easy to understand in Power BI: The model is simple, which helps developers and users to easily understand how data is connected.\n·       Faster report performance: Reports run faster because Power BI has fewer joins to
process and the model is clean.\n·       Reduce relationship complexity: One-to-many relationships between dimensions and fact tables make the model stable, clean and easy to maintain.\nDisadvantages of star schema\n·       Data redundancy: Dimension tables may have repeated information because the data is not highly normalized.\n·       Limited for operational reporting: When reports need very granular transaction-level data, the star-schema structure misses some detailed fields.\n·       More storage needed: Since the same data appears multiple times, the overall model size can be larger compared to snowflake schemas.\nWhat is a snowflake schema  \nA snowflake schema is another data-modeling technique where one central fact table connects to multiple dimension tables, but one or more of those dimension tables are further divided into multiple related sub-dimension tables, resulting in a structure that resembles a snowflake. Instead of having all descriptive information in a single dimension table, the data is split into multiple smaller tables based on logical categories. 

In the Power BI snowflake-schema model, the central fact table connects to the primary dimension tables using a one-to-many relationship just like a star schema; the subdimensions table is positioned on the “one” side  and the primary dimension table is positioned on the “many” side.
\n\n
                                                         \n Snowflake schema\nAdvantages of snowflake schema\n·       Removes data duplication: Data is stored in multiple related tables, so the same information is not repeated.\n·       Suitable for detailed reporting: Extra tables allow detailed information while keeping the main dimension clean and simple.\n·       Saves storage space: The model is more normalized; it uses less space compared to a star schema.\nDisadvantages of snowflake schema\n·       Slower performance: Because data is split into many small tables, Power BI needs more joins, which can slow down the report.\n·       More complex model: A snowflake schema has multiple linked tables, which makes the structure harder to understand than a simple star schema.\n·       Not ideal for large fact tables: When fact tables are large, too many joins from snowflake dimensions can affect performance.\nPerformance in Power BI: Star vs snowflake
\nPerformance is one of the biggest differences between the two schemas:\n·       The star schema performs faster because it has fewer tables and fewer joins. Power BI’s VertiPaq engine compresses star-schema tables more efficiently, which results in faster visual, faster DAX calculations and better memory usage.\n·       Snowflake schema is slower because Power BI needs to combine many small tables using relationships. Every additional join adds processing overhead, which can affect query performance.\nIn most real-world Power BI solutions, the star schema is preferred for performance.

So how do you decide when to use which schema in Power BI?  Use a star schema when:\n·       You want fast report performance \n·       Data is not highly detailed or transactional\n·       You want a simple model that business users can understand\n·       You are building dashboards for executives or business teams\n·       Performance is more important that storage optimization\n
Use a snowflake schema when:\n·       Your data is highly detailed and needs multiple descriptive columns\n·       You want to avoid data duplication\n·       Storage space is a concern\n·       You have many hierarchical attributes \n
\nConclusion\n
Both star and snowflake schemas have their own strengths for data modeling.

The star schema is easy to understand, performs faster, and is usually the best choice for dashboards that need quick results. On the other hand, the snowflake schema is helpful when the data needs to be well organized, detailed and highly structured.

The best schema depends on what your project needs. If your goal is smooth performance and a clean model, the star schema is the right fit. But if your data is complex and requires proper normalization, the snowflake schema can offer better control and clarity.

In the end, a carefully planned data model leads to faster reports, better insights and a more efficient Power BI solution giving users a clear and reliable reporting experience.


Power BI Data Modeling: Star vs Snowflake Schema

By \nPraveen Reddy Atturi, HEXstream senior data analytics engineer\nOracle Integration Cloud (OIC) connects applications and data sources in the cloud, while Oracle BI Publisher (BIP) generates reports from that data. Integration of OIC with BIP allows you to automatically trigger reports, pass parameters and deliver the output to other systems, reducing manual effort and ensuring timely access to critical information.\nPrerequisites:\nBefore beginning setting up the integration, make sure you have the following ready:\nBI Report Path\n·     The full catalog path of the BI Publisher report you want to call.\nReport XML Output\n·     Run the BI report once and download the XML output.\n·     This file will later be converted into an XSD schema for mapping.\nWSDL URL of BI Publisher\n·     The SOAP WSDL used to invoke the BI Publisher report.\nUser Credentials\n·     User ID and password of the Oracle Fusion application with permission to run the BI report.\nOpaque.xsd Schema File\n·     Used to manage Base64 encoded report output before decoding.\nXSD Schema of Report XML Output\n·     Generate an XSD from the BI report’s XML output file.\n·     This schema will be uploaded into OIC to parse and map report data.\nFirst, we need to have an existing BIP report so that we can get the full path of that BI Publisher report, which uniquely identifies the report within the BI Publisher catalog and is mandatory for OIC to call it. Then we require XML output which will later be converted into an XSD schema for parsing and mapping. Afterwards we must also have the WSDL URL of the BI Publisher web service, as this enables OIC to invoke the report through the SOAP adapter. \nAdditionally, make sure you have valid Oracle Fusion user credentials (user ID and password) with permission to run the report, since authentication is required. Alongside this, the opaque.xsd schema file is needed to manage the report output returned by BI Publisher. Finally, generate the XSD schema from the report’s XML output, which OIC will use to understand the data structure and map fields correctly to the target system.\nSteps to Integrate OIC with BI Publisher:\nCreate REST Adapter Connection\n·      To start the integration.\nCreate SOAP Adapter Connection\n·      To call the BI Report.\nCreate Integration\nAdd REST Adapter Connection\n·      Add JSON Request format.\n·      Add JSON Response format!\nAdd SOAP Adapter Connection\n·      Configure the SOAP Adapter Connection (between start and end\n·      Set the report path and parameters.\nStart Mapping for SOAP Adapter Connection\nCreate Write Stage File\n·      This will store the BI Report output data (decoded).\nExample:\n§  File name: Filename.xml.\n§  File path: /filepath.\n·      Upload the opaque.xsd file.\nMap Write Stage File with SOAP Adapter Mapper\n·      Map the report byte or SOAP response with opaque schema.\nCreate Read Stage File \n(from ICS file reference)\n·       Reads data from the first stage file (decoded data).\n·       Add file reference from Write Stage File (ICS file reference).\n·       Upload the BI Report output XML.xsd file.\nMap Read Stage File with End Point (Target)\n·     Map each column from the BI Report response to the endpoint fields.\nFirstly, BI Publisher (BIP) reports must be available before integrating with OIC. These reports should be fully developed, tested, and optimized for performance and output format, as they serve as the data extraction and formatting engine. OIC does not design or generate the reports. Instead, it functions as the control layer that triggers the report, passes parameters, receives the output, and delivers the data to downstream systems.\nIntegrating Oracle Integration Cloud (OIC) with BI Publisher (BIP) involves a structured process that begins with creating a REST adapter connection in OIC. This acts as the trigger for the integration. External applications or systems can call this connection by sending a JSON request with the required parameters, which will then kick off the entire process.\nOnce the REST adapter is set up, the next step is to create a SOAP adapter connection. This adapter is responsible for calling the BI Publisher report. By configuring it with the report’s WSDL and setting the proper report path and parameters, OIC can request and retrieve the report output from BI Publisher.\nAfter both adapters are set up, create a new integration in OIC. For this task, an App Driven Integration type works best. Inside the integration, add the REST adapter first and specify the request and response formats in JSON so data can move in and out of OIC correctly.\nNext, add the SOAP adapter to the flow. Place it between the start and end points of the integration and configure it with the BI Publisher report details. At this stage, make sure the SOAP adapter has the right report path and that all parameters are defined. Once this is done, map the input values from the REST adapter to the SOAP adapter so that the report gets executed with the correct parameters.\nWhen the SOAP response comes back, it will usually contain the report output in Base64 format. To handle this, you need to create a Write Stage File. This stage file will temporarily store the decoded report output. Define a file name and path for this file, such as Filename.xml and /filepath, and upload the opaque.xsd schema. The opaque schema is used to correctly handle the Base64 encoded content.\nNow map the SOAP response to the Write Stage File using the opaque schema. This process converts the encoded data into a usable XML file. To process the XML, create a Read Stage File that points to the Write Stage File. Upload the BI Publisher report’s XML XSD schema here. This schema describes the structure of the report output and helps OIC
understand the hierarchy, elements, and fields.\nFinally, map the fields from the Read Stage File to the target endpoint. Each column from the BI Publisher response should be linked to the corresponding field in the target system. This ensures the data flows smoothly and can be used by the receiving application without any issues.\nBy following these steps, you can build a reliable integration between OIC and BI Publisher. The setup may look detailed, but once it is configured correctly, the entire process runs automatically. Reports can be triggered through a REST call, executed via SOAP, decoded, stored, and delivered to downstream systems in the right format.\nThis approach eliminates manual effort, reduces errors, and ensures timely access to business-critical information. It is flexible, allowing adjustments to report parameters, output formats, or mappings without rebuilding the entire integration. Overall, integrating OIC with BI Publisher provides a scalable and reliable solution that makes reporting processes faster, more efficient, and easier to maintain.

Tech Corner Tutorial: Integrating OIC With BI Publisher

By Sirisha Patibandla, HEXstream data analyst\nIn today’s data-driven business world, turning raw transactional data into usable insights is essential. Oracle Fusion Data Intelligence (FDI) is Oracle’s advanced analytics platform that connects operational data with strategic decision-making. But what really happens behind the scenes—from when data is created in Oracle Fusion Cloud Applications to when it shows up in a dashboard?\nLet’s explore the FDI data pipeline and see how it provides reliable, timely, and relevant insights.\nWhat is Fusion Data Intelligence?\nOracle Fusion Data Intelligence is a cloud-based analytics solution that enhances Oracle Fusion Applications (ERP, HCM, SCM, CX) with ready-made data models, KPIs, and dashboards. It helps business users and analysts get real-time insights without needing to piece together data from different sources.\nAt the core of FDI is a robust data pipeline that handles data ingestion, transformation, data modeling, and visualization automatically. Understanding this pipeline is crucial for maximizing the benefits of FDI.\nThe anatomy of the FDI data pipeline\nHere’s how the pipeline works in practice:\n\n\nFigure: The five stages of the FDI data pipeline\n1. Data Extraction from Oracle Fusion Applications\nSource Systems: Oracle Fusion ERP, HCM, SCM, and CX modules\nMechanism: Data is extracted using Oracle’s secure and scalable framework, which uses REST APIs and Oracle Integration Cloud (OIC) connectors\nFrequency: Near real-time or scheduled batch loads, depending on the module and business needs\n2. Staging and Ingestion\nLanding Zone: Extracted data is first staged in a secure object store (e.g., Oracle Object Storage).\nIngestion Engine: Oracle Cloud Infrastructure (OCI) Data Integration or Oracle GoldenGate can be used to transfer data into the FDI warehouse.\nValidation: Data is checked for completeness, schema consistency, and referential integrity before progressing to the next stage.\n3. Data Transformation and Modeling\nTransformation Layer: FDI uses dbt (Data Build Tool) for managing SQL-based transformations, which include:\nData cleansing and enrichment\nApplying business logic\nResolving hierarchies\nSemantic Model: A curated semantic layer is created with Oracle Analytics Semantic Modeler, enabling user-friendly views of complex data.\n4. Data Warehouse and Storage\nWarehouse Engine: Oracle Autonomous Data Warehouse (ADW) is the foundation for FDI’s data storage.\nPartitioning and Indexing: Optimized for performance, the warehouse utilizes partitioning strategies and materialized views to support quick query execution.\nSecurity: Row-level security and role-based access control maintain data privacy and compliance.\n5. Visualization and Insight Delivery\nOracle Analytics Cloud (OAC): Users interact with data through prebuilt dashboards, ad hoc analysis, and natural language queries.\nEmbedded Analytics: FDI allows insights to be embedded directly into Fusion Applications, supporting contextual decision-making.\nAlerts and KPIs: Users can set up threshold-based alerts and monitor KPIs in real time.\nFeatures that strengthen the pipeline\nBeyond the core stages, FDI integrates enhancements that boost usability and ensure data integrity.\nMachine Learning Integration: FDI supports predictive analytics using
     Oracle Machine Learning (OML) on ADW.\nData Lineage and Governance: Built-in tools ensure transparency in
     data flow and transformation logic.\nSelf-Service Extensions: Business users can expand the semantic
     model and create custom metrics without needing IT help.\nFrom raw data to real decisions\nFDI’s pipeline isn’t just technical—it drives business outcomes.\nFinance Example: A manager reviews supplier payment delay.
     FDI extracts invoice and payment data, transforms it to compute average
     payment cycles, and displays supplier-level dashboards. The manager
     identifies a trend, sets up alerts, and initiates a policy change—all
     within the same platform.\nHR Example: An HR leader analyzes employee attrition. FDI pulls workforce data
     from Fusion HCM, applies predictive models, and highlights departments at
     risk. This enables proactive retention strategies.\nWhy it matters\nSpeed to Insight: Ready-made models and automation shorten time to value.\nTrust and Accuracy: Controlled data models ensure consistency across departments.\nScalability: Built on OCI, FDI can grow with enterprise data volumes and complexity.\nConclusion\nOracle Fusion Data Intelligence is more than a reporting tool—it’s a strategic asset. By understanding how its data pipeline works, organizations can better recognize the value it provides and make informed choices about customization, governance, and adoption.\nWhether a data engineer, business analyst, or enterprise architect, mastering the FDI pipeline is the key to unlocking the full potential of Oracle Fusion Applications.\n\nCLICK HERE TO CONTACT US ABOUT YOUR DATA PIPELINE.\n 
\n

From Fusion To Forecast: Navigating The FDI Data Pipeline

\nBy Ramaswamy Iyappan, HEXstream Python developer\nLLMs are evolving from “chatbots” to multi-step, tool-using agents. But one question shows up repeatedly when designing real-world AI systems:\nShould retrieval happen automatically through tool calling? Or should we run RAG manually and stuff the retrieved context into the prompt?\nMost teams (us included) accidentally mix these two paradigms, run into unpredictability, and later realize that each method has a very different purpose.\nThis article breaks down both patterns, compares them deeply, and provides a clear framework for knowing when to use which—with cost, latency, examples, and architectures.\n\nTwo retrieval paradigms: The core difference\nIn agent design, “retrieval” can happen in one of two ways.\n\nA. Context stuffing (maual RAG / deterministic retrieval)\nYou perform the retrieval (vector search, keyword, SQL, hybrid)
 → You decide what top-k chunks to send
 → You build the prompt with something like:\nSystem: ...\nContext: <docs here>\nUser: <question>\nThe LLM has no say in retrieval. It gets the context already stuffed in.\n✔ Most common in RAG pipelines
✔ Stable, predictable behavior
✔ One LLM call\nThis is what most enterprises use today for Q&A, knowledge bots, and SQL generation.\n\nB. Tool calling (retrieval as a function)\nThe model receives no upfront context. Instead, it is given tools such as:\nsearch_docs(query) -> list[Document]\nrun_sql(query) -> rows\nget_user_profile()\n\nThe LLM decides:\n●     IF it needs to call retrieval\n●     WHAT query to search for\n●     HOW many times\n●     WHEN to stop

\nThis is how \nagents\n operate in frameworks like LangGraph, LangChain, OpenAI Tools, and MCP.\n✔ Better reasoning
✔ Self-directed
✔ More flexible workflows\nBut also:\n✘ More expensive
✘ More variable
✘ Harder to predict/evaluate\n\nThe mistakes we made—and how we fixed it\nIn one of our earlier implementations:\n●     We built an AI system that always requires domain context\n●     But we exposed retrieval as a tool (optional)\n●     So the LLM \nsometimes skipped retrieval\n, sometimes hallucinated, sometimes over-called tools

(5–6 retrieval calls per question)\nThe result?\n●     Token cost went up\n●     Latency spiked\n●     Hallucinations increased\n●     Tool-calling behavior was inconsistent\nThe fix:\n👉 For tasks that always need context, retrieval should NOT be a tool.
👉 We moved to deterministic Context Stuffing for those flows.
👉 We reserved tool calling for true “actions”—SQL execution, conversions, plotting, etc.\nThis stabilized everything: cost, behavior, accuracy.\n\n\nArchitectural difference \nContext stuffing pipeline (determinstic RAG)\nUser Query\n   ↓\nRetriever (vector search / SQL search)\n   ↓\nRe-ranker\n   ↓\nSelect top-k chunks\n   ↓\nPrompt = Instructions + Context + Query\n   ↓\nLLM (one call)\n\n●     Knowledge Q&A\n●     SQL generation\n●     Documentation bots\n●     Internal enterprise assistants\n●     Product support\n●     Any domain where grounding is always required\n\nTool calling pipeline (LLM-decides retrieval)\nUser Query  \n   ↓  \nLLM  \n   → Decide if context needed  \n       → Call search_docs()  \n       → Inspect results  \n       → Possibly call again  \n   → Create final answer  \nBest for: \n●     Multi-step reasoning\n●     Agents with goals\n●     Tool composition workflows\n●     Decision-making\n●     “Act on the world” logic (APIs, SQL execution, RPA-like tasks)\n\n\nToken usage & cost implications \nContext stuffing (manualr RAG)\n●     One LLM call\n●     Context length predictable (top-k × chunk size)\n●     Cheap, especially with context compression\n●     Easy to cache retrieval results\nTypical cost: \nPrompt tokens = system + query + context\nCompletion tokens = final answer\nTotal calls = 1\n\n\nTool calling retrieval \n●     Each tool call incurs:

\n○     A reasoning step (LLM call)\n○     A tool result (tokens)\n○     Another LLM step for the next decision\nTypical sequence\nCall 1 → decide to retrieve \nCall 2 → interpret retrieved docs  \nCall 3 → generate final answer\n→≈ 2–4× cost

→ More latency

→ More variance (sometimes more tool calls than necessary)\n\nWhen to prefer which (the real framework)\nHere is the clean decision table we ended up using internally.\n\nUse content stuffing when: \n\n●     SQL generation\n●     Policy explanation\n●     Product documentation\n●     Codebase Q&A\nHallucination is unacceptable\n●     Enterprise data assistants\n●     Analytics reporting\n●     Medical or legal text explainers\nLatency must be low\n●     SaaS user-facing chatbots\n●     Real-time operations dashboards\nYou want stable evaluation\nSame (query + context) → same answer

No tool-calling randomness.\n\nUse tool calling when: \nRetrieval is conditional\n\n●     “Get me the latest revenue number”\n●     “Find customers in California”\n●     “Fetch the schema for table X”\nActions are required \n●     Run SQL\n●     Call external APIs\n●     Create Jira tickets\n●     Perform calculations\n●     Upload files\nYou want autnomous multi-step reasoning\n●     Research agents\n●     Planning agents\n●     Multi-hop workflows\n●     Systems that chain 3–10 steps\n\nConcrete examples: SQL agent\nIncorrect (what we did initially): \n●     Give LLM a tool: search_docs(schema_query)\n●     Ask it to decide when to retrieve schema context\n●     Hope it retrieves the right tables before generating SQL\nResult:\n●     Sometimes retrieves wrong docs\n●     Sometimes retrieves nothing\n●     Sometimes over-retrieves\n●     SQL had hallucinated columns\n\nCorrect (what we fixed):\n●     Always run deterministic RAG:\n○     Retrieve 5 columns per table\n○     Re-rank\n○     Provide final structured context\nBuild:

System + SQL style guidelines\nContext: schema columns + joins + examples\nUser query\n●     Only expose SQL execution as a tool\nResult:\n●     No hallucinations\n●     Faster\n●     Cheaper\n●     Predictable\nHybrid pattern (the most powerful) \nIn mature systems, the best pattern is:\nUse context stuffing for knowledge grounding\n(RAG → prompt context)\nUse tool calling for actions / structured data\n(run_sql, search_db, call_api,
send_notification)\nThis mirrors deep-learning research on “modular agents”:

RAG = memory

Tools = actions

LLM = policy layer\n\nFinal summary table\nFeature\nContext
  Stuffing (Manual RAG)\nTool
  Calling\nRetrieval triggered by\nYOU\nLLM\nToken cost\nLow\nHigh\nLatency\nLow\nMedium/High\nHallucination risk\nMinimal\nMedium\nIdeal for\nKnowledge tasks\nAction tasks\nDeterminism\nHigh\nLow\nNumber of LLM calls\n1\n2–6\nEvaluation\nEasy\nHard\nBest used when\nContext always required\nContext sometimes required\n\nFinal thoughts\nRetrieval is not just a technical utility—it defines your agent’s entire behavior style.\nContext Stuffing = grounded, predictable, cost-efficient
Tool Calling = flexible, autonomous, multi-step reasoning\nUsing the right one in the right place is the difference between a slow, unpredictable agent and an enterprise-grade production AI system.\n\n\n


Tool Calling vs. Context Stuffing: Choosing The Right Retrieval Strategy In AI Agent Design

By Sirisha Patibandla, HEXstream data analyst\nAs businesses generate ever-increasing volumes of data, from customer interactions to IoT devices, managing and analysing it efficiently has become a major challenge. Traditional data-management systems often struggle to keep up with this data flood, requiring manual tasks like infrastructure management, data migration, and constant database optimization.\nThis is where the Autonomous Data Warehouse (ADW) comes into picture.\nWithout the usual hassles of manual database administration, the Autonomous Data Warehouse offers an innovative method for businesses to store, manage and analyze their data in a world where flexibility and real-time access to data are essential for success.\nWhat is an Autonomous Data Warehouse?\nAn Autonomous Data Warehouse is a cloud-based data-management solution that automates the repetitive operations using artificial intelligence (AI) and machine learning (ML). By minimizing the need for manual intervention in operations like provisioning, patching, backups, scalability and security, ADWs allow organisations to focus on what fundamentally matters: extracting insights and driving better decisions.\nKey features\nAutomation: ADWs use AI and ML to automate standard database-administration operations, such as scaling, patching, backups and provisioning. This minimizes human errors and saves time by reducing the need for manual intervention.\nSelf-tuning: ADWs use real-time usage patterns to automatically optimize database performance. Without the need for manual tuning, machine learning continuously adjusts indexing, query performance, and other parameters to ensure effective functioning.\nBuilt-in security: ADWs offer strong security that shields data in transit and at rest without the need for further human configuration through encryption, automatic threat detection, and compliance features.\nScalability: ADWs can automatically scale up or down in response to fluctuations in workload, allowing them the flexibility to handle data surges and ensuring steady performance while minimizing expenses.\nSelf-repairing: ADWs detect, diagnose and resolve issues autonomously, assuring minimal down-time and uninterrupted service. They monitor for anomalies, automate containment, and fixes, and apply updates automatically. Predictive maintenance anticipates potential issues (such storage capacity approaching) and takes preventative measures to avoid interruptions.\nArchitecture of Autonomous Data Warehouses\nAt its core, an ADW combines cloud infrastructure, AI-driven automation, and secure storage into a unified system: \n\n\n
Layered architecture of an Autonomous Data Warehouse\nStorage layer: Elastic, cloud-based storage that scales seamlessly with data growth.\nCompute layer: Autonomous compute resources that adjust dynamically to workload demands.\nAI/ML layer: Embedded intelligence for tuning, repairing, and optimizing performance.\nSecurity layer: End-to-end encryption, role-based access, and compliance monitoring.\nUser interface: A self-service, intuitive interface that enables both technical and non-technical users to query and analyze data.\nBenefits of ADWs\nCost efficiency: ADWs significantly lower operational expenses by efficiently allocating resources and eliminating the need for dedicated database administrators. Businesses of all sizes can benefit from this cost-effective solution as they only need to pay for the computing and storage resources they utilise.\nImproved performance: With automated tuning and optimization, ADWs ensure faster query performance, resulting in faster data retrieval and processing.\nEase of use: With its user-friendly interface, ADWs make data management and analysis easier and requires less technical knowledge. The self-service features make it accessible for both technical and non-technical users to access and analyze data.\nReal-time use cases\nAutonomous data warehouses are being adopted across various industries to solve diverse challenges. Here are a few examples:\nFinancial analytics: Banks and financial institutions handle vast amounts of transaction data, detect fraud in real-time, and meet regulatory requirements.\nRetail: Retailers analyze customer buying patterns, manage inventory, and personalize  marketing strategies.\nManufacturing: Manufacturers optimize their supply chain operations, monitor equipment health, and improve product quality using autonomous data warehouses.\nHealthcare: Hospitals and research institutions use ADWs to analyze patient records, predict disease risks, and accelerate clinical research—all while maintaining strict compliance with privacy regulations.\nChallenges and considerations\nData-privacy concerns: Increased autonomy can create concerns about privacy and security, especially in sensitive sectors.\nDependency on AI models: The effectiveness of an autonomous system depends heavily on the quality of the AI/ML models running it.\nCost of transition: Migrating from traditional data warehouses to autonomous systems requires significant investment and planning, which may not be feasible for smaller organizations.\nThe road ahead\nThe future of data management looks promising with ADWs at the forefront. As these systems continue to evolve, we can expect even more advanced features and capabilities, such as:\n·      Predictive analytics for proactive decision-making\n·      Natural-language queries for user friendly access\n·      Seamless integration with other AI-driven technologies\nConclusion\nAutonomous Data Warehouses represent a major leap forward in data-management technology. By automating routine tasks, enhancing security, and providing agility, they empower businesses to focus on what truly matters—extracting value from their data driving innovation.\nAs adoption of ADWs grow, we can anticipate a future where data management is more efficient, secure and intelligent than ever before. \n\nCLICK HERE TO CONTACT US ABOUT OPTIMZING YOUR DATA WAREHOUSING. \n

Autonomous Data Warehouses: The Next Frontier In Data Management

By Swathi Ambati, HEXstream data solutions engineer\nUtilities operate in a highly regulated environment that demands transparency, financial accuracy, and strong internal controls. As organizations modernize their operations on Oracle Fusion, Oracle Fusion Data Intelligence (FDI) has become an essential platform for delivering deeper insights, supporting audit readiness, and enabling data-driven governance.\nMany organizations transitioning from legacy ERP systems finds that while Oracle Fusion provides robust functionality, adapting to new reporting structures, terminology and subject-area designs can take time. Although FDI offers several out-of-the-box dashboards, business users often need adjustments or enhanced analytical capabilities tailored to their specific auditing and operational requirements. \nThis is where custom enhancements and subject-area extensions become especially valuable.\nThis article highlights practical use cases showing how utilities can use FDI for audit and compliance. It also points out situations where adding extra attributes such as banking details, policy-related fields, exception indicators, and procurement data can help create more complete and accurate analysis.\nEnhancing financial transparency with extended banking and policy-related information\nAuditing within utility organizations often requires detailed visibility into financial operations. To support this, additional attributes such as banking information, policy-driven markers, or organization-specific financial indicators may need to be incorporated into relevant FDI subject areas.\nThese enhancements help auditors validate:\nProper mapping of bank accounts to business units\nFinancial processes aligned with internal policies\nCompliance with organizational or regulatory guidelines\nBy surfacing these enriched attributes within FDI, internal and external auditors can trace financial activity more efficiently and verify process adherence without navigating multiple transactional screens in Oracle Fusion.\nExtending subject areas to capture missing or cross-system data\nOut-of-the-box FDI subject areas may not include every attribute needed for deep audit analysis. To bridge this gap, organizations can extend subject areas.\nWhen data originates from external systems, data augmentation can be used to bring that information into FDI, enabling unified and consistent reporting. These extensions enable organizations to:\nAnalyze data in one place rather than comparing multiple application views\nMinimize manual reconciliation efforts\nMake informed decisions using consolidated, audit-ready datasets\nAuditors and compliance teams can analyze data across:\nTime periods\nSupplier and vendor attributes\nBusiness units\nInvoice, receipt and transaction details\nAccount and cost-center structures\nThis provides a transparent and comprehensive view of financial and operational activities,
supporting exception monitoring, policy validation, and trend analysis.\nStrengthening procurement and receiving compliance audits\nProcurement processes in utilities involve several touchpoint purchase orders, receipts, invoices and approval workflows. By extending FDI subject areas with additional procurement- and receiving-related attributes, organizations can better monitor and audit:\nOrder creation and approval processes\nReceipt validations and exception thresholds\nThe PO → receipt → invoice lifecycle\nSupplier performance and adherence to expectations\nThese enhanced datasets give auditors end-to-end visibility into procurement activities, helping them identify control gaps, detect irregularities, and spot deviations that may indicate inefficiencies or compliance risks.\nWhy these enhancements matter for utilities\nUtility organizations face unique audit and operational challenges, including:\nHigh transaction volumes\nStrict compliance and regulatory standards\nMulti-layered process flows\nFinancial exposure across operational, procurement and purchasing channels\nBy extending FDI subject areas and integrating custom attributes, utilities can:\nStrengthens internal controls\nReduce audit cycle time\nImprove the accuracy of compliance and financial reporting\nGain better visibility into spending and operational behaviors\nDetect anomalies earlier and with greater confidence\nFDI effectively becomes a centralized audit and analysis platform for financial teams, compliance teams, and operational leaders.\nConclusion\nOracle Fusion Data Intelligence helps utilities improve audit readiness, financial transparency, and overall compliance. By adding extra banking details, policy-related fields, procurement information, exception indicators, and other important data into extended subject areas, utilities can get clearer insights and make better decisions.\nAs the utilities industry continues to grow and change, FDI will remain a key tool for maintaining financial accuracy, simplifying audits, and supporting responsible, data-driven operations across the organization.\n\nWANT TO BOLSTER YOUR AUDIT READINESS? CLICK HERE TO CONTACT US.\n 

Improving Audit Readiness For Utilities Using Oracle FDI

By Steven Hor, HEXstream data solutions engineer\nWhat is a Service Point? While a point of service is a common terminology in people’s mind, this question technically depends on what modules are enabled within Customer-2-Meter (C2M) implementations. \nWith multiple edge applications, C2M can be implemented with the following modules as one singular system:\nCustomer Care & Billing (CCB)\nMeter Data Management (MDM) \nOperational Device Management (ODM)\n Work and Asset Management  (WAM)\nFor each module, there is a primary table referencing its own entity of Service Point and can be deployed independently.\nWhen Oracle Utility Analytics (OUA) is deployed 0ut-of-the-box (OOTB) with C2M, Service Point Dimension is dedicated for MDM use and is likely broken for other modules. Consequently, the ID for Service Point will be referenced as D1_SP driven by the MDM table D1_SP. As a result, data warehouse would need to be aware of this to ensure proper Extract, Transform and Load (ETL) into a star-schema data structure. \nIn perspective, the Service Point dimension table is normalized and leveled with a 1:1 relationship across other modules. This table requires a unique ID and cannot be duplicated due to cardinality and integrity. Since there are multiple references of Service Point across multiple modules, designers and architects must use caution when retrieving data fields from MDM to CCB or MDM to ODM/WAM. To achieve this, the table D1_SP_IDENTIFIER will contain ID to join with CI_SP (CCB SP table) and W1_NODE (ODM/WAM Node table). In C2M, once a SP is created or updated, the ID will be populated across the enabled modules and data will be updated upon.\n\nWANT TO IMPROVE YOUR NAVIGATION IN C2M? CLICK HERE TO CONTACT US. \n

Tech Corner Tutorial: OUA Service Point Dimension With C2M

By Bharath Moola, HEXstream data solutions engineer\nEvery modern enterprise runs on data. From millions of customer interactions to machine readings, from financial transactions to operational records, information moves constantly across systems and platforms. The real challenge is not how to collect this data, but how to keep it accurate, consistent and available everywhere all at once.

This is where data-replication tools make their impact. They ensure that data in one system matches data in every other system, keeping business operations aligned in real time. These tools may not always be visible, but they are essential for keeping insights current, reports accurate, and applications synchronized. Without replication, a company’s digital heartbeat would lose its rhythm.\nLet's explore some data-replication tools...\nOracle GoldenGate\nOracle GoldenGate remains one of the most trusted names in enterprise data replication. It works behind the scenes, capturing every change at the database log level and delivering it wherever it needs to go, almost instantly. Its reliability has made it a preferred solution for global banks, utilities and telecommunications companies that require uninterrupted data flow.

GoldenGate’s strength lies in its ability to maintain availability even during upgrades or unexpected outages. It supports replication across multiple platforms and geographic regions, allowing businesses to maintain continuous operations with minimal downtime. Deep integration with Oracle Database and Oracle Cloud Infrastructure enables seamless data movement between on-premise systems and the cloud.\nQlik Replicate\nQlik Replicate is designed to simplify complex data movement. With its user-friendly interface, teams can create and monitor replication pipelines without extensive coding or configuration. The platform supports a wide variety of databases and cloud technologies such as Snowflake, Databricks and Azure Synapse.

Its continuous streaming and agentless design help reduce latency and maintain steady performance. For organizations that operate both legacy systems and modern cloud environments, Qlik Replicate offers an efficient bridge to real-time analytics and unified visibility across data landscapes.\nHevo Data\nHevo Data is built for automation and scalability. Developed as a cloud-native platform, it provides no-code data pipelines that can stream information from more than one hundred and fifty sources into destinations such as BigQuery, Redshift and Snowflake.

Hevo automatically manages schema changes, transformations and error handling, creating a reliable and self-maintaining data flow. Businesses use it to eliminate manual ETL work and ensure that data remains accurate and current. By automating these repetitive processes, Hevo allows data teams to focus on analysis and innovation rather than maintenance.\nAWS Database Migration Service\nAWS Database Migration Service, or DMS, helps organizations move data to the cloud without disruption. It continuously replicates data from on-premises systems into Amazon’s cloud databases such as RDS, Aurora and Redshift. The service handles monitoring, failover and scaling automatically, reducing the workload for technical teams.

DMS is compatible with major databases including Oracle, SQL Server, MySQL and PostgreSQL, which makes it a natural choice for hybrid cloud environments. Its flexibility and reliability help enterprises modernize infrastructure and maintain real-time synchronization as they transition to the cloud.\nFivetran\nFivetran focuses on keeping analytics continuously refreshed. It provides a fully managed and automated solution that extracts and loads data from hundreds of systems into cloud-data warehouses. The platform ensures that dashboards and BI tools are always working with the latest information.

Automatic schema recognition, real-time updates, and built-in reliability make Fivetran a popular choice among analytics teams using platforms such as Snowflake, Databricks and BigQuery. It removes the complexity of managing pipelines and allows teams to access accurate data without delay.\nThe future of data replication\nData replication is evolving beyond synchronization. Artificial intelligence and automation are beginning to predict replication issues before they happen and repair broken connections without human intervention. Event-driven replication will allow organizations to trigger actions in real time whenever data changes, turning traditional reporting into continuous, intelligent decision-making.

In the near future, replication will not only keep systems in sync but also help enterprises adapt instantly to changes across operations. It will serve as the foundation for predictive analytics, digital twins, and autonomous decision systems.

Replication may run quietly in the background, but it powers everything that defines a modern enterprise. The right mix of tools such as Oracle GoldenGate for precision, Qlik Replicate for flexibility, Hevo Data for automation, AWS DMS for modernization, and Fivetran for analytics readiness transforms data from static information into a living, intelligent asset that drives business forward.\n


The Best Data-Replication Tools Powering Real-Time Enterprise Intelligence

By Sunil Koparde, HEXstream project manager\nAI is no longer just a buzzword—it has become a core part of modern products and processes. \nYet for many people, AI still means \nChatGPT\n or conversational tools. In reality, AI has been around for years, quietly powering products and processes through recommendation engines, predictive analytics, smart chatbots, and automation. Developers have long been embedding intelligence into their code, and product teams have consistently pushed for smarter, faster, and more adaptive systems.\nA product manager’s job isn’t to build AI models or fine-tune algorithms, but is to ensure that when AI is introduced into a product, it solves a real problem, delivers business value, and integrates seamlessly into the user experience. Similarly, managing AI projects differs from traditional software initiatives—processes, people, risks and even success metrics are evolving.\n1.       Start with “Why AI?” not  “Where can we add AI?”\nThe first temptation many teams face is to add AI just to appear innovative. That’s the wrong starting point. A product manager’s first responsibility is clarity: \nWhat specific user or business problem does AI solve better than existing methods?\nAI should enhance capability, not complexity. For example:\n·       Predicting demand trends faster than manual analysis\n·        Personalizing recommendations to improve user engagement\n·       Automating repetitive support queries to save time\nThe “why” defines everything else—the data needed, the architecture, the success metrics, and how users interact with the feature.\n2.       Understand the shift: From logic to learning\nTraditional software development is based on logic. Developers write clear rules: \nIf X happens, do Y. \nAI, in contrast, learns from data, which means outcomes are probabilistic, not deterministic. This shift has two major implications for product managers: First, you manage model behavior, not fixed rules. Secondly, you must account for data quality and change over time. \nAs product managers, we must adjust our mindset: success isn’t about completing requirements—it’s about training, monitoring, and improving performance and accuracy over time.\n3.      New team dynamics and roles\nAI projects introduce new stakeholders. Besides engineering, design and QA, you’ll now coordinate with:\n·       Data scientists who develop and train models\n·       Machine-learning engineers who operationalize the models\n·       Data engineers who build and maintain data pipelines\n·       Legal and compliance teams who review ethics, bias and privacy risks\nA product manager’s job becomes one of translation—ensuring everyone understands why the AI exists and what success looks like. They will need to connect business intent with technical design in clear, measurable terms.\n4.      Data is the foundation\nIn AI, data is not an afterthought—it \nis\n the product. Without quality data, even the best models fail. And from a product manager’s viewpoint, this introduces new responsibilities:\n·       Assess data availability early. \nDo we have enough relevant, high-quality data to train the model?\n·       Plan for data collection. \nIf not, how will we gather it—ethically and legally?\n·       Monitor for bias. \nDoes the dataset represent diverse user groups fairly?\n·       Evaluate privacy. \nAre we respecting regulations and user consent?\nPreparing data takes longer than building the model itself. Planning for that time is part of smart product management.\n5.      Defining the right success metrics\nAI metrics differ from traditional KPIs. You need two perspectives:\n·       Model-performance metrics (accuracy, precision, recall)\n·       User or business outcomes (engagement, satisfaction, ROI)\nFor example, a chatbot might have 95% intent-recognition accuracy, but if users still prefer to call support, the model’s success doesn’t translate into real product value. A product manager must define metrics that measure impact, not just intelligence.\n6.      Managing uncertainty and iteration\nAI development is inherently experimental. Models improve over time with data, so product managers must embrace iteration. Here’s what that means in practice:\n·       Launch in limited scope or beta\n·       Observe real-world performance\n·       Collect new data\n·       Retrain or fine-tune models regularly\nIt’s similar to agile delivery—but instead of just releasing code updates, you’re releasing and retraining intelligence.\n7.      Ethics, fairness and transparency\nAI introduces ethical considerations that go beyond functionality. A product manager needs to ask:\n·       Can users trust the system’s decisions?\n·       Are results fair across different demographics?\n·       Are users aware when AI is making choices for them?\nTransparency matters. For example, in the utility industry, if an AI system assigns a particular crew to a job, users deserve to understand \nwhy.\n8.      Cost, ROI and sustainability\nAI systems can be expensive tobuild and maintain. Training models, storing large datasets, and running inference at scale all add operational costs. As such, product managers must evaluate costs to develop and deploy. They must predict expected business return (ROI) and gauge ongoing maintenance efforts.\nSometimes, a simple automation or rule-based approach can achieve the same result much faster and cheaper. The product manager’s role is to challenge assumptions and choose the solution that maximizes impact...not the one that sounds most sophisticated.\nProduct manager’s evolving role\nManaging AI projects redefines the productmanager’s role in several ways:\n·       You think beyond features to systems that learn\n·       You measure not just delivery, but accuracy and trust\n·       You collaborate across a broader, more technical ecosystem\n·       You become a bridge between data and decision-making\nIn the AI era, product managers evolve from feature managers to value architects. They understand that adding AI into products is not just about technology—it’s about direction, discipline and responsibility. Increasingly, a product manager’s job is to:\n·       Identify where AI truly adds value\n·       Ensure ethical and transparent implementation\n·       Manage uncertainty through iteration\n·       Balance cost, capability and ROI\n·       Keep users at the heart of every decision\nAI can transform products—this fact becomes clearer each day. But that transformation is optimized when it is guided by thoughtful, informed product management. \n\nWANT MORE? CLICK HERE TO CONNECT WITH US ABOUT USING AI TO MANAGE YOUR PROJECTS.\n \n


Why AI? Product Management & Artificial Intelligence Projects For Utilities

By Ramaswamy Iyappan, HEXstream software development engineer\nIn the age of AI-powered analytics, one fundamental challenge persists: how do we teach artificial intelligence to understand the complex, enterprise-grade databases that power modern businesses? \nAt HEXstream, we've been exploring how to solve this problem for GenBI, an upcoming AI product that aims to transform natural language queries into SQL and deliver intelligent insights from enterprise databases. But before any AI can generate a single SQL query, it must solve a foundational problem: understanding your database schema.\nThis isn't just about reading table names and column definitions. Enterprise databases are complex ecosystems with hundreds of tables, intricate relationships, views, materialized views, synonyms, and constraints. Traditional approaches fall short. We need solutions that can automatically discover, index, and semantically search through vast database schemas in real-time.\nIn this blog post, we'll explore the conceptual challenges of schema discovery for AI systems, the approaches that work, and why this foundational problem is critical for the future of AI-powered analytics.\nEnterprise databases are vast. A typical enterprise database might contain:\n●     Hundreds of tables with thousands of columns\n●     Complex relationships defined through foreign keys, constraints, and business logic\n●     Multiple abstraction layers including views, materialized views, and synonyms\n●     Sparse documentation where column comments and table descriptions are often missing or outdated\nWhen you're dealing with hundreds of tables and thousands of columns, simply listing them all is not a viable approach. The AI needs to understand what's relevant to each specific query, not just what exists in the database.\nWhen a user asks "Show me customers who made purchases in the last month," the AI needs to: \nIdentify which tables contain customer and purchase data\nUnderstand the relationships between these tables\nKnow which columns represent dates, amounts, and customer identifiers\nUnderstand business rules and constraints that affect the query\nThe challenge is that this information is often scattered across the database schema. Customer data might be in a table called CUSTOMERS, CLIENTS, CUST_MASTER, or even TBL_USR_001. Traditional keyword matching fails when naming conventions vary or when business terminology differs from technical naming.\nPerhaps the biggest challenge is the semantic gap between how humans think about data and how databases store it. A user might think in terms of "customers" and "purchases," but the database might have:\n●     Tables named with technical abbreviations (CUST_TXN, ORD_HDR)\n●     Columns with cryptic names (CUST_ID, ORD_DT, AMT)\n●     Relationships that aren't explicitly defined in constraints\n●     Business logic embedded in views or application code\nBridging this semantic gap requires more than just reading schema metadata—it requires understanding the meaning and purpose of database objects.\nQuerying schema information from database system tables for every user request is expensive:\n●      System table queries can take seconds on large databases\n●     Processing and embedding thousands of schema objects is computationally intensive\n●     Loading full schema information into LLM context windows is token-prohibitive (can easily exceed 50,000 tokens)\nWe need solutions that can retrieve relevant schema information in milliseconds, not seconds, while staying within token budgets that allow for efficient AI processing.\nThe solution to these challenges lies in a multi-layered approach that combines:\nAutomatic Schema Extraction from database system tables\nVector Database Indexing for semantic search\nIntelligent Caching for performance optimization\nManual Context Enrichment for bridging the AI-human gap (required)\nLet's explore each layer conceptually.\nModern databases maintain comprehensive metadata about their structure in system tables (also called data dictionary views or information schema). These system tables contain information about:\n●      Tables and their columns\n●      Data types and constraints\n●      Relationships between tables (foreign keys)\n●      Views, materialized views, and synonyms\n●      Indexes and other database objects\nThe first layer of schema discovery involves automatically querying these system tables to extract a complete picture of the database structure. This extraction must handle:\n●     Multiple object types: Not just tables, but also views, materialized views, and synonyms\n●     Relationships: Foreign keys, primary keys, and other constraints that define how tables relate\n●     Metadata: Column comments, data types with precision, and other descriptive information\n●     Schema evolution: Detecting when schemas change and updating the extracted information\nAutomatic extraction eliminates the need for manual schema documentation. Instead of requiring database administrators to maintain separate documentation (which often becomes outdated), the system can always work with the current state of the database.\nHowever, extraction alone isn't enough. The extracted schema is just raw metadata—it tells us what exists, but not what's relevant to a specific query. That's where the next layer comes in.\nOnce we've extracted the schema, we face a new challenge: how do we quickly find the relevant tables and columns for a user's natural language query?\nTraditional keyword search falls short. A user might ask about "customer transactions," but the database might have tables named CUST_TXN, CUSTOMER_ORDERS, or CLIENT_PURCHASES. We need semantic understanding, not just string matching.\nVector databases solve this problem by converting schema objects (tables, columns, constraints) into high-dimensional vectors called embeddings. These embeddings capture semantic meaning, not just keywords.\nHere's how it works conceptually:\nEmbedding Generation: Each schema object (table, column, constraint) is converted into a vector using an embedding model. These models are trained to understand that "customer" and "client" are semantically similar, even though they're different words.\nVector Storage: These embeddings are stored in a vector database optimized for fast
     similarity search.\nSemantic Search: When a user asks a question, the query is also converted into an embedding. The vector database then finds the most semantically similar schema objects, regardless of exact keyword matches.\nVector databases are specifically designed for this use case. They provide:\n●     Fast similarity search: Can find similar vectors in milliseconds, even with millions of objects\n●      Semantic understanding: Understands meaning, not just keywords\n●     Scalability: Can handle large numbers of schema objects efficiently\n●     Metadata filtering: Can combine semantic search with metadata filtering (e.g., "find tables related to 'customer' that have more than 10 columns")\nSchema extraction and vector search are expensive operations:\n●     Querying system tables can take seconds\n●     Generating embeddings requires API calls to embedding models\n●     Vector search, while fast, still has overhead\nFor an AI system that needs to respond in real-time, these operations must be optimized through intelligent caching.\nAn effective caching strategy uses multiple levels:\nIn-Memory Schema Cache: Caches the raw schema data extracted from database system tables. This eliminates repeated system table queries.\nVector Index Cache: The vector database itself serves as a cache, storing embeddings so they don't need to be regenerated.\nRetrieval Result Cache: Caches the results of schema retrieval queries, so identical or similar queries don't require new searches.\nCaching introduces a new challenge: When should the cache be invalidated?\nThe cache must be invalidated when:\n●     Schema changes are detected (new tables, modified columns, etc.)\n●     User manually triggers a refresh\n●     Application restarts (cache is rebuilt)\nHowever, detecting schema changes itself can be expensive. Effective solutions balance the cost of change detection with the cost of serving stale data.\nEffective caching can provide dramatic performance improvements:\n●     Schema Extraction: From seconds to milliseconds for cached data\n●     Vector Search: Sub-millisecond search on indexed data\n●     Token Usage: 80-90% reduction by retrieving only relevant schema objects\nWhile automatic schema extraction, vector indexing, and caching provide the technical foundation, there's a critical gap that automation alone cannot bridge: the context gap between what the database contains and how humans actually use it.\nA database might contain a table called TBL_ORD_001 with columns ORD_ID, ORD_DT, and ORD_AMT. The automatic extraction can tell the AI that this table exists and what its structure is. The vector database can help find it when someone asks about "orders." But the AI still doesn't know:\n●     What business process this table represents\n●     How it's typically used in queries\n●     What business rules govern its data\n●     What relationships exist beyond foreign key constraints\n●     What edge cases or special considerations apply\nSimply connecting an AI to a database is not enough. Automatic schema discovery provides the structure, but not the meaning. To bridge the gap between AI understanding and human knowledge, manual context enrichment is not optional—it's a must-required step.\nHere's why:\nBusiness Context: Database schemas don't capture business logic, domain knowledge, or how data is actually used in practice. A column might be named STATUS, but only domain experts know that "A" means "Active" and "I" means "Inactive," or that certain statuses are deprecated but still in the database for historical reasons.\nUsage Patterns: Automatic discovery can't tell the AI which tables are frequently used together, which columns are most important for common queries, or which views are preferred over base tables for certain use cases.\nSemantic Understanding: Technical naming conventions often don't align with business terminology. A table might be called CUST_MASTER, but in the business, it's referred to as "customer accounts" or "client records." Manual enrichment bridges this semantic gap.\nData Quality Context: Databases don't document data quality issues, known problems, or special handling requirements. Only manual context can tell the AI that "this column sometimes contains nulls for legacy records" or "this view filters out test data."\nRelationship Context: While foreign keys define technical relationships, they don't capture business relationships. Two tables might not have a foreign key constraint but are frequently joined based on business logic. Manual context captures these implicit relationships.\nManual context enrichment involves cleaning, preparing, and presenting your data in an understandable way to the AI. This process includes:\nAdding meaningful descriptions to columns helps the AI understand what each column represents. For example:\n●     Instead of just CUST_ID, add a comment: "Unique customer identifier, used for joining with order tables"\n●     Instead of just ORD_STATUS, add: "Order status: 'P' = Pending, 'C' = Completed, 'X' = Cancelled"\nProviding high-level summaries of what each table contains and how it's used:\n●     "Customer master table containing all registered customers. Used as the primary source for customer information across the system."\n●     "Order header table storing order-level information. Typically joined with order line items table for complete order details."\nCreating business-friendly aliases for technical column names:\n●     CUST_ID → alias: "Customer ID"\n●     ORD_DT → alias: "Order Date"\n●     ORD_AMT → alias: "Order Amount"\nDocumenting business rules that aren't captured in database constraints:\n●     "Orders with status 'X' should not be included in revenue calculations"\n●     "Customer records with REGION = 'TEST' are test data and should be excluded from production reports"\n●     "The CUSTOMERS table and CLIENT_ACCOUNTS table represent the same entity but from different systems; use CUSTOMERS as the source of truth"\nDocumenting how tables and views are typically used:\n●     "Use the SALES_SUMMARY view instead of joining SALES and PRODUCTS tables directly"\n●     "The CUSTOMER_ORDERS view is preferred for customer order queries as it includes necessary filters and calculations"\n●     "When querying order data, always join with CUSTOMERS table using CUST_ID, not CUSTOMER_NAME"\nDocumenting relationships beyond foreign key constraints:\n●     "The ORDERS table relates to CUSTOMERS via CUST_ID, but also has an implicit relationship with SHIPMENTS via ORDER_ID (no foreign key constraint exists)"\n●     "The PRODUCTS and INVENTORY tables should be joined on PROD_ID, but only include rows where INVENTORY.QTY > 0"\nManual context enrichment cannot be fully automated because:\nDomain Knowledge: Understanding business context requires domain expertise that exists in human minds, not in database schemas.\nOrganizational Context: Each organization uses its databases differently. What works for one company's schema might not apply to another, even if the structures are similar.\nEvolving Understanding: Business requirements and data usage patterns evolve over time. Manual enrichment allows organizations to document this evolving understanding.\nQuality Control: Manual enrichment ensures that the context provided to the AI is accurate and reflects actual business practices, not assumptions.\nWhile manual context enrichment requires upfront effort, it's an investment that pays significant dividends:\nImproved AI Accuracy: Better context leads to more accurate SQL generation and fewer errors.\nReduced Iteration: When the AI has good context, it's more likely to generate correct queries on the first attempt, reducing back-and-forth with users.\nBetter User Experience: Users get better results faster when the AI understands their data properly.\nScalability: Once context is enriched, it benefits all users and all queries. The investment scales across the entire organization.\nKnowledge Preservation: Manual enrichment captures organizational knowledge that might otherwise be lost when domain experts leave or documentation becomes outdated.\nAutomatic schema discovery provides the foundation, but manual context enrichment provides the bridge. You cannot avoid the manual flow of adding as much database context as possible—it's a must-required step to first clean, prepare, and present your data in an understandable way to the AI.\nThe goal is to provide as much context as possible so the AI can make informed decisions. This context is what transforms a technical database schema into a business-understandable data model that the AI can effectively work with.\nSchema discovery is the foundation that enables natural language to SQL conversion. Without understanding the database schema, an AI system cannot:\n●     Identify which tables contain the data needed to answer a query\n●      Understand how tables relate to each other\n●     Generate accurate SQL queries with correct column names and join conditions\n●     Optimize queries based on available indexes and constraints\nEffective schema discovery enables:\nDemocratized Data Access: Non-technical users can query databases using natural language, without needing to understand SQL or database structure.\nFaster Insights: Users can get answers to data questions in seconds, not hours or days.\nReduced Errors: AI-generated SQL is more likely to be correct when the AI understands the schema accurately.\nScalability: Systems can work with large, complex databases without manual
     configuration.\nAdaptability: Systems can adapt to schema changes without manual reconfiguration.\nFuture systems might learn schema patterns from successful queries:\n●     Join patterns: Which tables are frequently joined together?\n●     Column usage: Which columns are commonly queried together?\n●     Business rules: Discovered relationships not captured in constraints.\n●     User preferences: Column aliases and descriptions preferred by users.\nExtending support beyond single database types:\n●     Multiple database engines: Support for PostgreSQL, MySQL, SQL Server, etc.\n●     Data warehouses: Support for Snowflake, BigQuery, Redshift\n●     Hybrid systems: Support for databases with mixed schemas\nBuilding an AI that truly understands enterprise databases requires solving a fundamental problem: schema discovery. This isn't just about reading table names and column definitions—it's about creating a system that can:\nAutomatically extract comprehensive schema information from databases\nSemantically understand the meaning and purpose of database objects\nEfficiently retrieve relevant schema information for each query\nEnrich with manual context to bridge the gap between technical schema and business understanding (required)\nContinuously adapt to schema changes and user feedback\nThe solution lies in a multi-layered approach that combines:\n●     Automatic schema extraction from database system tables\n●     Vector database indexing for semantic search\n●     Intelligent caching for performance optimization\n●     Manual context enrichment for bridging the AI-human gap (a must-required step)\nThis foundational capability enables a new generation of AI-powered analytics tools that can answer complex questions in natural language, democratize data access, and transform how businesses interact with their data.\nAt HEXstream, we're exploring these concepts as we develop GenBI, an upcoming AI product that aims to bring the power of natural language to enterprise database analytics. The challenges are significant, but the potential impact is transformative.\nThe future of business intelligence is AI-powered, and it starts with understanding your data. Schema discovery is the foundation that makes this possible.\nThis blog post explores conceptual approaches to schema discovery for AI systems. The concepts discussed are based on industry best practices and ongoing research in AI-powered analytics. \nClick here to connect with HEXstream to learn more. \n


Build AI That Understands Your Data: The Challenge Of Enterprise Database Schema Discovery

By Kiran Boddireddy, HEXstream Data Solutions Engineer\nIn today’s data-driven energy landscape, power utilities generate vast amounts of information—from smart-meter readings and SCADA logs to outage reports and compliance records. Managing this data effectively isn’t just about storage; it’s about optimizing performance, cutting costs, and maintaining regulatory compliance.\nThis blog explores modern data-archiving strategies tailored for power utilities, featuring the best tools and techniques. Let’s take a look…\n1. Cloud-based archiving for smart grid & meter data\nPower utilities deal with huge volumes of real-time data from smart meters, SCADA systems, and outage-monitoring tools. Keeping all of this in active databases can quickly drive up costs and slow down performance.\nCloud-based archiving offers a scalable, cost-effective alternative—retaining data securely while keeping it accessible when needed.\nBest tools for power utilities:\nAWS S3 Glacier: Archival for SCADA and smart-meter logs\nGoogle Cloud Coldline: Secure storage for outage history\nMicrosoft Azure Blob Archive: Ideal for compliance and billing records\n2. On-premises archiving for sensitive grid data\nFor utilities managing sensitive grid operations, on-premises archiving ensures tighter control, faster access, and higher security.\nBest tools for power utilities:\nHitachi Content Platform: Stores substation logs and transformer-health data\nNetApp StorageGRID: Hybrid archiving for distribution data\nDell EMC PowerScale: Retains load forecasting and energy-demand trends\n3. Hybrid storage for compliance and long-term retention\nPower utilities must comply with FERC, NERC and ISO/RTO regulations while managing decades of operational and customer data. Hybrid storage balances cost, accessibility, and compliance needs.\nBest tools for power utilities:\nVeritas Enterprise Vault: Automates archiving for regulatory data\nCommvault: Handles hybrid storage for consumption and failure records\nIBM Spectrum Protect: Stores maintenance logs for high-voltage assets\n4. Automated data-lifecycle management\nAutomation can transform how utilities manage data. With automated lifecycle policies, older data seamlessly moves from high-cost storage to cold archives, saving money without manual effort.\nBest tools for power utilities:\nAWS S3 Lifecycle Policies: Automates SCADA-log migration\nGoogle Cloud Data Lifecycle: Moves older energy-consumption data\nAzure Storage Lifecycle: Archives historical energy-price data\n5. Database archiving for utility operations\nAs databases grow, performance suffers. Archiving outdated operational records—such as power usage or grid telemetry—improves database speed and reduces maintenance costs.\nBest tools for power utilities:\nOracle Database Lifecycle Management: Archives billing and grid logs\nSAP ILM (Information Lifecycle Management): Retains historical-generation data\nIBM Db2 Archive: Manages compliance and demand-trend storage\n6. Tape storage for historical power-grid records\nFor data that must be retained for decades—such as power generation or outage history—tape storage remains the most cost-effective long-term solution.\nBest tools for power utilities:\nLTO (Linear Tape-Open): Low-cost archival for outage data\nIBM TS4500 Tape Library: Retains up to 50 years of historical energy data\nSpectra Logic Tape Storage: Ideal for plant-operation backups\n7. AI & machine learning for smart-data archiving\nArtificial intelligence is reshaping data management. AI-powered archiving helps utilities automatically classify, predict and move data—minimizing human error and unnecessary storage costs.\nBest tools for power utilities:\nAWS Macie: Classifies and archives low-priority data\nGoogle Cloud AI for Data Management: Identifies critical vs. redundant grid data\nAzure AI Data Governance: Drives compliance and security archiving\nChoosing the right archiving method\nMethod\nBest For\nKey Benefits\nCloud Archiving\nSmart meter, SCADA, billing data\nScalable, cost-effective\nOn-Premise Storage\nGrid telemetry, transformer logs\nFast access, strong security\nHybrid Storage\nCompliance, historical data\nCost & accessibility balance\nAutomated Lifecycle Mgmt\nRedundant operational data\nHands-free cost savings\nDatabase Archiving\nCustomer & energy usage data\nFaster queries, lower maintenance\nTape Storage\n10+ years of legacy data\nUltra-low-cost long-term retention\nAI-Based Archiving\nPredictive maintenance, optimization\nSmart automation, lower costs\nFinal thoughts\nFor modern power utilities, a hybrid archiving strategy is the key to success—combining cloud scalability, on-premises security, and automated lifecycle management.\nBy choosing the right mix of tools and methods, utilities can:\nReduce storage costs dramatically\nEnhance grid efficiency\nStay compliant with evolving regulations\nData is the new power—but how you store and manage it determines how efficiently you deliver energy to the world. 


Smart Data Archiving Strategies For Utilities: Cutting Costs, Boosting Efficiency & Staying Compliant

By Ashwin Kota, HEXstream data solutions engineer\n
For utility industries, delivering reliable power and minimizing outages is a constant challenge. With massive volumes of operational and customer data, leveraging machine learning (ML) directly within the database can transform how reliability is monitored, predicted, and managed. OML4SQL allows utility analysts to perform predictive, descriptive, and anomaly analytics directly within Oracle SQL—eliminating the need to move sensitive data to external tools.\nWhat is OML4SQL?\n
OML4SQL is an embedded machine learning platform inside Oracle Database that lets you apply models, score data, and perform analytics directly using SQL commands. Analysts can run predictive models, detect anomalies, cluster feeders or customers, and generate insights without leaving the database. \nKey keatures for utility industries:\nPredictive analytics: Forecast potential outages, identify high-risk feeders or device failures.\nDescriptive analytics: Cluster feeders by performance, detect unusual outage patterns, summarise reliability metrics like SAIDI, SAIFI, CAIDI, MAIFI.\nOn-the-fly scoring: Perform analytics dynamically without pre-training a model (exploratory mode).\nScalability & security: Computation happens entirely inside Oracle Database, ensuring high performance and data security.\nInstalling and setting up OML4SQL\n
OML4SQL is available for Oracle Database versions 12c (12.2) and higher including 18c, 19c, 21c. It is often pre-installed in Autonomous Database environments, but for on-premises setups you’ll need to ensure certain components and privileges are in place.\nDatabase requirements\nOracle Database version 12.2 or higher.\nThe Oracle Data Mining (ODM) component (underlying OML4SQL) must be installed. \nRequired privileges for the user: ability to execute DBMS_DATA_MINING packages, create models, and access data tables.\nInstallation steps\nCheck if OML4SQL (ODM) is installed\nSELECT comp_name, status \nFROM dba_registry \nWHERE comp_id = 'ODM';\nStatus should be VALID.\nInstall/Enable ODM if required\n/rdbms/admin/odminst.sql\nCreate a user for ML operations\nCREATE USER oml_user IDENTIFIED BY password;\nGRANT CONNECT, RESOURCE TO oml_user;\nGRANT EXECUTE ON DBMS_DATA_MINING TO oml_user;\nGRANT SELECT ANY TABLE TO oml_user;\nVerify installation\nSELECT * FROM ALL_MINING_MODELS;\nYou should see model meta-data and definitions.\nIn Autonomous Database, the OML4SQL engine is pre-installed; you may only need to enable relevant features.\nCore functions of OML4SQL\nKey functions you will use in OML4SQL:\n·      PREDICTION(model USING columns): Returns predicted class or value.\n·      PREDICTION_PROBABILITY(model USING columns): Returns probability/confidence of the prediction.\n·      CLUSTER_ID(model USING columns): Returns the cluster number/ID for given input row.\n·      MODEL_DETAILS(model): Use to inspect model definitions and metadata.\nPractical ML examples for utility reliability analytics\nBelow are queries that emphasise ML usage (prediction, clustering, anomaly detection) using tables related to Outage Restoration Steps and Outage Feeder Summary i.e. OUTAGE_RESTORATION_STEPS_FACT and OUTAGE_FEEDER_SUMMARY_FACT.\n1. Predicting high-risk feeders\nSELECT feeder_name,\nPREDICTION(outage_risk_model USING ci,
cmo, momentary_flag, m_ci) AS risk_class,\n PREDICTION_PROBABILITY(outage_risk_model
USING ci, cmo, momentary_flag, m_ci) AS risk_score\nFROM outage_feeder_summary_fact\nWHERE outage_start_dttm >= SYSDATE - 30\nORDER BY risk_score DESC;\nHere, a model named outage_risk_model predicts which feeders are likely to have an elevated outage risk in the next period.\n2. Classifying momentary vs. sustained outages\nSELECT outage_id, feeder_name,\nPREDICTION(outage_type_model USING
duration_mm, cust_restored, momentary_flag) AS outage_type,\nPREDICTION_PROBABILITY(outage_type_model
USING duration_mm, cust_restored, momentary_flag) AS type_confidence\nFROM outage_restoration_steps_fact\nWHERE rel_calc_begin_time >= SYSDATE - 30;\nThe model outage_type_model classifies each outage step as either “Momentary” or “Sustained” based on features like duration, customers restored, momentary flag.\n3. Clustering feeders by reliability metrics\nSELECT feeder_name,\nCLUSTER_ID(feeder_cluster_model USING
saidi, saifi, maifi) AS cluster_group\nFROM outage_feeder_summary_fact;\nUsing a clustering model feeder_cluster_model, this query segments feeders into groups by their reliability performance (SAIDI, SAIFI, MAIFI). Useful for targeted maintenance planning.\n4. Detecting anomalies in restoration steps\nSELECT outage_id, feeder_name,\nPREDICTION(anomaly_model USING
duration_mm, cust_restored, cust_remaining) AS anomaly_flag,\nPREDICTION_PROBABILITY(anomaly_model
USING duration_mm, cust_restored, cust_remaining) AS anomaly_score\nFROM outage_restoration_steps_fact\nWHERE rel_calc_begin_time >= SYSDATE - 15\nORDER BY anomaly_score DESC;\nHere the model anomaly_model flags restoration steps with unusual behaviour (high durations, unexpected customer restores)—an early warning for reliability engineers.\n5. On-the-fly predictive scoring (dynamic)\nSELECT feeder_name,PREDICTION_PROBABILITY(1 USING *) OVER () AS likely_high_risk\nFROM outage_feeder_summary_fact;\nThis query uses dynamic scoring without specifying a model — useful for exploratory trending when you don’t yet have a fully trained model.\nUse cases in utility reliability\n·      Outage risk prediction: Predict which feeders or devices will likely cause outages before they happen.\n·      Outage classification: Classify outage types (momentary vs sustained) for better resource allocation.\n·      Reliability segmentation: Cluster feeders based on performance metrics for targeted
interventions.\n·      Anomaly detection: Identify unusual restoration events or durations that deviate from
norms.\n·      Real-time scoring & exploration: Use dynamic scoring to explore emerging patterns without waiting for model build.\nWhy OML4SQL is ideal for utilities\n1.    No Data Movement: Everything happens inside Oracle, so sensitive operational data remains secure, and performance is high. \n2.    Familiar SQL Interface: Analysts can use ML via SQL functions they already use, lowering
adoption barriers.\n3.    Scalable for Big Data: Handles very large tables of outage events, feeders, customer counts. \n4.    Actionable Insights: Embeds intelligence directly in SQL queries so analytics become part ofoperational workflows.\nQuick start cheat sheet for OML4SQL in utility reliability analytics\n1. Core prediction functions\nFunction\nPurpose\nExample\nPREDICTION(model USING columns)\nPredict class or numeric outcome\nPREDICTION(outage_risk_model USING ci, cmo, momentary_flag)\nPREDICTION_PROBABILITY(model USING columns)\nConfidence / probability of prediction\nPREDICTION_PROBABILITY(outage_risk_model USING ci, cmo,
  momentary_flag)\nCLUSTER_ID(model USING columns)\nAssign cluster group based on model\nCLUSTER_ID(feeder_cluster_model USING saidi, saifi, maifi)\nMODEL_DETAILS(model)\nInspect model metadata\nSELECT MODEL_DETAILS('outage_risk_model') FROM dual;\n2. Common use cases & sample queries\nUse Case\nSample SQL\nPredict high-risk feeders\nSELECT feeder_name,
  PREDICTION_PROBABILITY(outage_risk_model USING ci, cmo, momentary_flag) AS
  risk_score FROM outage_feeder_summary_fact;\nClassify outage type\nSELECT outage_id,
  PREDICTION(outage_type_model USING duration_mm, cust_restored,
  momentary_flag) AS outage_type FROM outage_restoration_steps_fact;\nCluster feeders\nSELECT feeder_name, CLUSTER_ID(feeder_cluster_model
  USING saidi, saifi, maifi) AS cluster_group FROM outage_feeder_summary_fact;\nDetect anomalies\nSELECT outage_id,
  PREDICTION(anomaly_model USING duration_mm, cust_restored, cust_remaining) AS
  anomaly_flag FROM outage_restoration_steps_fact WHERE rel_calc_begin_time
  >= SYSDATE - 15;\nDynamic scoring\nSELECT feeder_name,
  PREDICTION_PROBABILITY(1 USING *) OVER () AS likely_high_risk FROM
  outage_feeder_summary_fact;\n3. Useful analytical tips\n·      Use the OVER() clause for on-the-fly scoring without explicitly defined model.\n·      Ensure feature columns such as ci, cmo, momentary_flag, m_ci, duration_mm are
cleansed and consistently formatted.\n·      Choose the right algorithm for the task (decision tree for classification, clustering for segmentation, anomaly detection for restoration outliers) — OML4SQL supports many.\n·      Leverage model metadata through MODEL_DETAILS() to understand predictor importance and model behaviour.\n·      Embed predictions into operational workflows (dashboards, alerts, maintenance planning) rather than keeping them in isolation.\nSummary\n

OML4SQL empowers utility industries to move from reactive reporting to proactive outage management. By integrating ML directly into SQL, teams can predict outages, detect anomalies, cluster feeders, and optimize service reliability within a secure, scalable database environment.\n

How Oracle SQL And AI Can Improve Analytics Insights Using OML4SQL

By Bharath Moola, HEXstream data solutions engineer\nUtilities are transforming faster than ever. From smart meters and IoT sensors to renewable integration and outage analytics, data now drives every part of the business. But as the flow of information grows, the real challenge is choosing the right platform to manage it. \nThree cloud players dominate this discussion: Oracle Cloud Infrastructure (OCI), Snowflake, and Databricks. Each has its own strengths and understanding them can help utilities turn raw data into real-time intelligence.\nOracle Cloud Infrastructure (OCI): Power through integration\nUtilities have long relied on Oracle for mission-critical systems like customer information, outage management and billing. OCI takes that legacy and extends it into a unified cloud environment. It connects operational data, analytics and AI so teams across planning, engineering and customer service can work from one source of truth.\nThe Autonomous Database reduces manual tuning, while the AI Data Platform introduces built-in machine learning and vector search for predictive insights. Utilities can use these capabilities to forecast demand, detect asset failures, or analyze outage trends without building separate systems. Because OCI integrates easily with existing Oracle applications,
modernization happens with less disruption.\nOCI is the reliable choice for organizations that value control, security and seamless integration across their operational and analytical layers.\nSnowflake: Simplicity and speed\nSnowflake has earned its reputation as the easiest cloud warehouse to use. It separates compute and storage, enabling utilities to scale up when meter or sensor data surges and scale down when usage drops. That elasticity is ideal for handling AMI, SCADA and customer-analytics workloads.\nAnalysts can query massive datasets using familiar SQL while the platform handles performance automatically. For leadership dashboards and compliance reporting, Snowflake delivers quick, clean results. It also supports modern workloads through Snowpark, which brings Python and AI capabilities directly into the platform.\nSnowflake is a favorite for utilities that want fast deployment, self-service analytics, and minimal maintenance. It excels when teams prioritize speed and simplicity over heavy customization.\nDatabricks: Innovation at scale\nDatabricks is built for experimentation and advanced analytics. It unifies data engineering, machine learning, and streaming in one platform. Utilities use it to process massive IoT streams, predict equipment failure, and optimize grid operations with AI models.\nIts Delta Lake technology ensures reliability and governance across huge data lakes. Engineers and data scientists can collaborate in shared notebooks, building predictive models that combine weather data, sensor readings and asset histories.\nDatabricks offers power and flexibility for utilities pushing into real-time analytics and AI. It demands more setup and management than Snowflake, but it rewards that effort with deep control and creative freedom.\nComparison at a glance\nCapability\nOracle Cloud Infrastructure (OCI)\nSnowflake\nDatabricks\nCore Focus\nUnified data, analytics, and AI in one enterprise platform\nCloud-native
warehouse built for analytics and reporting\nOpen
lakehouse combining data engineering, AI, and streaming\nIdeal Use Case\nIntegration
with Oracle CIS, OMS, AMI, and asset systems\nFast analytics, regulatory reports, and BI dashboards\nReal-time
processing, machine learning, and predictive maintenance\nEase of Use\nModerate,
with strong automation through Autonomous Database\nVery easy
and intuitive, minimal infrastructure management\nModerate
to complex, requires engineering expertise\nAI/ML Features\nBuilt-in AI, vector search, and semantic enrichment\nSnowpark
for Python-based AI workloads and Cortex AI features\nMLflow for
model lifecycle management, collaborative notebooks, and streaming pipelines\nScalability\nHigh scalability across Oracle Cloud regions and services\nElastic,
independent scaling of compute and storage\nHigh, based on cluster configuration and autoscaling\nGovernance and Security\nEnterprise-grade,
unified identity and policy controls\nMature
governance with role-based access and data sharing controls\nFlexible,
customizable governance with Unity Catalog\nBest Fit For\nEnterprise-wide
modernization leveraging Oracle applications\nAnalytics
teams seeking quick insights with minimal setup\nData scientists and engineers building AI and real-time solutions\nThe right platform for the right purpose\nChoosing between these platforms is not about picking a winner, but rather aligning technology with business goals.\nOCI suits utilities that want a single, governed environment linking operations, analytics and AI.\nSnowflake is perfect for organizations focused on data accessibility and clean reporting.\nDatabricks is for teams exploring advanced analytics and real-time intelligence.\nMany utilities combine them. For instance, a company might use Databricks for data processing and forecasting, Snowflake for analytics and dashboards, and OCI for integration with Oracle-based operational systems. This hybrid model provides both innovation and stability.\nPowering the utility of tomorrow\nThe future of utilities lies in data that can act, not just inform. Whether it’s predicting outages, balancing renewable generation, or personalizing energy services, the right data platform makes all the difference.\nOCI brings strength and integration. Snowflakes deliver clarity and speed. Databricks enables creativity and intelligence. Together, they define the cloud power play that will keep utilities reliable, efficient and ready for the next generation of smart energy.\n\nNEED HELP WITH YOUR CLOUD MIGRATION? CLICK HERE TO CONTACT US. \n

The Cloud Power Play: Choosing Between OCI, Snowflake And Databricks For Smarter Utilities

By Devika Gayapu, HEXstream data solutions engineer\nYour applications generate a constant flood of data, from customer activity and outage data to IoT-sensor readings and website clicks. Before long, all that information starts piling up, and you need a way to store it that’s quick to access, easy to manage, and doesn’t eat up too much space.\nThis is where the choice between row-based and columnar-file formats really matters. Both hold the same data, but the way they organize that data can completely change how you store, query and analyze it.\nLet’s explore…\nRow-based storage\nIn row-based storage, data is saved record-by-record. Think of it like filling out a paper form for each person—writing their name, age and city together. The next person’s form starts right after the first one.\nThis method has been used for many years and works well for systems that handle lots of small, complete transactions such as processing payments or storing customer profiles.\nThe pros of row-based storage:\nSimple and fast when you need to read or write whole records.\nWorks well for systems that deal with one record at a time.\nEasy to manage when applications always need the full record.\nThe cons of row-based storage:\nSlower for analytics because it reads all columns even if you need only one or two.\nTakes more space because mixed data types do not compress as well.\nColumnar storage\nNow imagine sorting all the forms by column instead. You keep all names together, all the ages together, and all the cities together. That is columnar storage. In this setup, data is grouped and stored by column, which is great for analytics because most queries look at only a few columns across many rows.\nThe pros of columnar storage:\nMuch faster for analytics since it reads only the columns you need.\nUses less space because similar data types compress better.\nSpeeds up queries on large datasets by reducing how much data is scanned.\nThe cons of columnar storage:\nSlower for updates since one change can affect multiple column files.\nNot ideal when your system needs to frequently read or write full records.\nCommon big-data file formats\nHere are three popular file formats and when to use them.\n1. Avro (row-based): Avro stores data row-by-row. It is great for streaming data and real-time systems where you need to write full records quickly. It supports schema changes so you can later add or remove fields without issues. Avro is often used in data pipelines with tools like Kafka and Spark Streaming.\n2. Parquet (columnar): Parquet stores data by column and is designed for analytics and reporting. It efficiently compresses data and reads only what is needed, which makes queries much faster. Parquet is widely used in tools like Spark, Hive and AWS Athena for analyzing large datasets.\n3. ORC (Columnar): ORC also stores data by column and is optimized for Hadoop and Hive. It includes indexes and statistics that make queries even faster. ORC is useful when you work with large Hive tables or batch-processing jobs.\nWhich should you choose?\nIf your system writes or streams a lot of new data, Avro is a good choice because it handles fast-writes and allows flexible schema changes.\nIf your goal is to run analytics or reports on large amounts of data, Parquet works best because it compresses well and makes queries faster.\nIf your environment is based on Hive or Hadoop, ORC will give you better performance because it is built for those tools.\nIn summary, row-based formats like Avro are best when you need to frequently write or update complete records. Columnar formats like Parquet and ORC are better when you need to efficiently analyze large volumes of data.\nThe best choice depends on how you use your data. If you need faster writes, go with a row-based format. If you need faster reads for analytics, go with a columnar format. Choosing the right one will make your data systems faster, lighter and more cost-effective.\n\nCLICK HERE TO CONNECT WITH US ABOUT OPTIMIZING YOUR DATA-STORAGE STRATEGIES.\n \n


The Big Data Trio: Understanding Avro, Parquet And ORC In Simple Terms

By Krishna Sai Geetla, HEXstream data analytics engineer\n

\n\n⁠\nGoal: \nTo verify if a new dimension extracted through Data Augmentation feature can be added. \nKey points to consider:\n→ The new table must have matching primary key data to join with the out of the box dimension.\n→ Both the primary key names for existing dimension and new dimension must not be same\nFlow of steps:\n·      Custom dimension is brought through Data augmentation.\n·      In FDI admin condole select the data configuration option.
\n\n\n\n·      Select the Data source connector from which the new dimension is extracted.
\n\n\n\n\n·      Select Data augmentation.
\n\n\n\n\n·      Click on “create” dropdown in next window and select “augmentation”.
\n\n\n\n·      In the augmentation window, we can select augmentation type as create dimension> source dataset type is supplemental data and source table type is system provided.
\n\n
\n\n
\n\n\n\n\n\n\n\n·      Select the required source table from the connector that needs to be extracted as dimension in Target ADW and click on “Next”.
\n\n\n\n·      In the second window select the attributes for the dimension. 
\n\n\n\n·      Next window, select the primary key for the new dimension and click on “Next”.
\n\n\n·      Give a name for the Data augmentation. Then give table suffix which will be the physical suffix name of the table. Table name will be generated automatically according to the connector prefix. Lastly select “run immediately” in schedule and click on “finish”. Select -“continue” if subject area is not selected.
\n\n
\n\n\n·      Check the status in Data augmentation page.
\n\n\n·      To see the status of the extraction progress, go back to Data configurations page and select request history.
\n\n
\n\n
\n\n\n

\n·      After the job is completed, the next changes are in SME. Go to admin console and select Semantic model extensions.
\n\n\n\n·      Create a sandbox with a name.
\n\n\n\n·      In perform action, select manage logical star.
\n\n\n·      Select edit logical star> out of the box >Subject area name> fact name and click on “Next”.
\n\n\n·      In logical star page click on “Manage Dimension” and select “Add Dimension”.
\n\n\n·      Select the schema name. “OAX_USER” is default for DA tables. Select the object name which is the dimension name. 3rd Entry is automatically generated, use different name if the name is already used. Make sure there are no whitespace errors/ gaps before and after the name.\n·      Then select the attributes/columns of the custom dimension and primary key. Click “Next”.
\n\n


\n·      In the Next window uncheck hierarchy. Even though no hierarchy is selected, name must be given as mandatory step. In selected data elements tab, click “detail” to get the primary key details and click on edit display attribute to select the column.
\n\n\n

\n·      Click finish after all selections.
\n\n\n·      The new dimension is added to logical star > Join the new dimension to the fact.
\n\n\n\n·      Select the desired join type and give the primary keys of fact and the new custom dimension.\n·      The content level must be set to “Detail”.\n·      Click done as final step in logical star modification.
\n\n
\n\n

\n·      Go back to the sandbox window and click on “perform action” to “modify subject area”.
\n\n\n·      Select modify subject area and the required subject area and click “Next”.
\n\n\n·      Verify the folders of subject area and click “Next”.
\n\n\n·      In second wizard of subject area modification click “manage extensions” dropdown and select “Manage new customization”.
\n\n\n\n·      The new dimension folder must be visible and then the required column must be selected. All the folders will be added if we click on the first checkbox option beside the dimension name.\n·      Click on “Add”.
\n\n\n\n·      Verify the new folders in right tab and click on “Finish”.
\n\n\n\n·      Go back to user extensions page and apply changes to the sandbox.
\n\n\n·      Check the status in “activity” tab.
\n\n\n\n·      Publish the new sandbox changes from publish model option.\n\n\n\n·      Check the status in Activity Tab. \n\n\n\n\n·      After changes are published go to OAC Url or click on home page from FDI admin console.
\n\n\n·      Select the modified subject area from analytics dropdown> Data.\n\n\n\nCheck if the new dimension folder is added to the subject area\n\n\n\n·      Validate the data by selecting columns from the custom dimension.\n\n\n\n·      Check if the dimension is compatible with ootb fact by adding columns from the fact too or other existingdimensions. (This validates that there is join between custom dimension and
fact.)\n\n\n\nCLICK HERE TO CONNECT WITH US ABOUT OUDI. \n
\n

OUDI Use Case: Addition Of Non-OOTB Tables Through Data Augmentation

By Swathi Ambati, HEXstream data solutions engineer\nMany organizations face challenges with FERC regulatory reporting because financial data is scattered across multiple ledgers and affiliates. This fragmentation often causes errors and inconsistencies, making compliance and accuracy difficult to achieve. Manual reconciliation further slows reporting cycles, limiting the ability to deliver timely and reliable
filings.\nOracle Fusion Data Intelligence (FDI) addresses these challenges by consolidating data from multiple ledgers and affiliates into a single reporting layer. This ensures data accuracy, consistency, and compliance with FERC standards. Automated integration and aggregation simplify the handling of high-volume datasets, while streamlined reconciliation reduces cycle times and provides finance teams with real-time visibility.\nThrough a recently completed FERC project with a large utility in North America with multiple affiliates, HEXstream created a centralized data model using FDI, specifically designed to meet regulatory and financial requirements. Key reports delivered for this project include:\nFERC & GAAP Ledger Reporting:\n Consolidates primary and secondary ledgers, with detail available based on SLA configuration. Includes supplier, cost center, intercompany, project and task details, purchase orders, and inventory items.\nSubledger Cost Accounting:\n Provides comprehensive visibility into subledger entries, linking source transactions directly to the general ledger.\nBalance Sheet:\n Presents assets, liabilities and equity across ledgers and affiliates for accurate financial monitoring. Supports multiple levels of rollup and highlights differences between FERC and GAAP amounts.\nTrial Balance & Income Statement: \nDisplays balances and account details with drill-down to journals and subledger transactions, ensuring transparency, reconciliation and reporting accuracy.\nThese reports collectively provide comprehensive visibility into financial operations, streamline regulatory reporting, and empower affiliates to complete FERC forms accurately while gaining actionable financial insights for better decision-making. Data hub created in FDI now provides a comprehensive FERC reporting platform that significantly reduces manual work and makes FERC reporting quick and efficient. 

Using FDI To Simplify FERC Reporting

By Swathi Ambati, HEXstream data solutions engineer \nThe new \nOracle Utilities Data Intelligence platform (OUDI)\n offers tremendous optimizations to users. And like most powerful new solutions, we’ve encountered many questions as we introduce OUDI to utilities. \nBelow, we’ve compiled some of the most common questions related to OUDI, and we share our perspective in short responses. Take a look…\n1. Are the AMI, customer and device-data sources available out-of-the-box, specifically for the CCS data-source integration in OUDI?\n

Yes, these data sources (AMI, customer, device) are available out-of-the-box as part of the standard CCS (Customer Care and Billing System) integration within OUDI. They are preconfigured to support typical utility-data domains and come with base metadata and model mapping.\n2. What is the difference between the AMI, customer and device-data sources in OUDI? What types of objects are covered?\nAMI (Advanced Metering Infrastructure): Covers smart-meter data, interval readings, and meter events.\nCustomer: Includes customer profiles, accounts, contacts, service agreements, and
     related billing entities.\nDevice: Focuses on physical assets such as meters, transformers and sensors, including device types, status and configurations.\n3. Is there documentation or metadata available that outlines the tables or views associated with the AMI-data source?\nYes, there should be technical documentation provided by Oracle or your implementation team. This includes data-dictionary references or data-model diagrams that show table-to-entity relationships for each data source, including AMI. These are typically found in the OUDI Product Documentation Portal or the internal Confluence/SharePoint used by the product team.\n4. How can we inspect or explore the actual backend tables or views that are part of the AMI data source in OUDI?\n

You can explore the analytical base tables or stage tables within the ADW schema provisioned for OUDI; tools like SQL Developer can help inspect these tables. Schema  OAX_USER will contain the read-only tables.\n5. How are these three data sources (AMI, customer, device) connected to the semantic model in Oracle Analytics or ADW?\n

Each data source feeds into a corresponding set of analytical base tables in ADW, which are then extended into the semantic model. Subject areas in Oracle Analytics Cloud (OAC) or Fusion Data Intelligence (FDI) reference these base tables and define business-friendly measures and dimensions.\n6. Is the semantic layer in OUDI dynamically updated based on the selected data source (e.g., AMI)?\n

No, not dynamically in real-time. However, once the pipeline is configured and runs for a selected source, the metadata is registered and used to populate subject areas. Any changes to the data source or pipeline configuration may require a metadata refresh or remapping in the semantic layer.\n7. If we select AMI as the data source, does it alter downstream configurations such as pipeline settings or frequency?\n

Yes, the frequency and configuration may vary depending on the data source. For example, AMI data is typically ingested daily due to the high volume and time-sensitive nature of meter readings, whereas customer or device data might be updated less frequently.\n8. Is the data-ingestion pipeline frequency customizable for each data source?\n

Yes, OUDI allows you to configure frequency settings for each pipeline independently. This includes options such as daily, weekly or custom intervals depending on the operational requirements of each source system. \n9. Do all three data sources (AMI, customer, device) require independent pipeline setup and configuration?\n

Yes, each data source requires its own pipeline setup. While some configurations (like target schema or error handling) may be reusable, the source mappings, frequency and transformation logic will differ. This enables better control and optimization per source.\n\nINTERESTED IN OUDI? CLICK HERE TO CONNECT WITH US.\n\n


Common OUDI Questions, Unprecedented OUDI Capabilities

By Krishna Sai Geetla, HEXstream data analytics engineer\n\nLet's explore how to modify existing subject areas by creating metrics in existing subject areas. We can create a new metric in “My calculations” in OAC and verify results of both SME and OAC metric matched. \nGoals:
1) To verify if users can add a new calculation in existing subject area's fact by using simple calculations/limiting expressions and using columns from logical dimensions and the fact itself in the SME sandbox.
2) To verify if the same calculation created in SME can also be created as a custom calculation in the "my calculations" tab from the OAC subject area folder.
3) Ensure that the final verification is in reports whether the two metrics from SME and OAC yield the same result.

Flow of steps:\n·      In the FDI console click on Semantic model extensions.
\n\n


·      In user extensions click on create a sandbox.\n\n\n

·      Give a name to the sandbox and click on done.
\n\n\n·      After clicking done user will be taken to new window where we can manage logical star and subject area.\n·      Click on perform action and then on manage logical star.

\n\n\n·      As this use case is on existing subject area click on edit logical star>check on out of the box>select the subject area name>select the fact and click on next.
\n\n\n·      In the new window user can see the logical star structure.\n·      As we are adding a new metric on fact , right click on the fact and then on manage extension.
\n\n


\n\n·      In the next window click on manage dimension and select add columns in the drop down.
\n\n\n·      Click on the “add derived column” option.
\n\n\n·      Here we can add the content of the calculation. \n·      There are options to select analytical functions on the right side for any calculations or user can directly type in the code.
\n\n\n·      Here we are validating a calculation based on a filter.
\n\n\n\n\n·      To select which columns need to be added in the calculation, users must select from the dropdowns od current data elements to select from same fact or from other data elements where users can select from existing dimensions or calendar dimensions.
\n\n
\n\n\n·      To select a column from dimensions, users can click on logical tables> expand the required dimension and double click or drag and drop the desired column.
\n\n
\n\n
\n\n\n·      As this is a filter expression, we type in the limiting value for calculation as "Residential."
(Values in a dimension/fact cannot be selected. User must give the details.)
\n\n\n·      Entering name of the new metric is mandatory.
\n\n\n·      After giving name, check on the validate option to see the calculation expression is correct and click on save.
\n\n\n·      After clicking save, users are taken to other screens where metric details are present. Validate and click on finish.
\n\n\n·      Users can see the add col after clicking finish. After viewing, click on the back arrow.
\n\n\n·      Click back from the logical star-structure screen.\n\n\n\n·      Logical star changes are done now. \n·      Click on perform action to modify subject area with new changes.
\n\n\n·      Select manage subject area.
\n\n\n·      Select modify existing subject area> select the name > and click on next.
\n\n\n·      In the next wizard of the Modify Subject area, click on next after confirming that the folders of the existing subject area are going to be modified.
\n\n\n·      In the next wizard, click on the manage-elements drop down and select manage factory customizations.\n\n\n\n\n·      Check the new metric that must added to the subject area and click on add folders.
\n\n\n·      Select the fact name to add folder and click on add folders.
\n\n\n·      Cross check on the selected data elements with the new changes and click on finish.
\n\n\n·      After the subject area changes, go back to the previous page of user extensions.
\n\n\n\n\n·      Click on the sandbox options to apply changes.
\n\n\n\n·      Verify the task by going to activity page.
\n\n\n\n·      After changes are applied on sandbox, click on publish model and select the sandbox name. Then click on publish.
\n\n\n·      Check the task updates in Activity.
\n\n
\n\n\n\n·      After confirming the publishing of user extensions in Sandbox, go to OAC page by clicking on home page in Main FDI console or through OAC URL.
\n\n\n\n·      Click on the dropdown to select the data option.
\n\n
\n\n\n\n\n·      Search and select the subject area name.
\n\n\n\n·      Verify if the new column change is added to the folder.
\n\n\n·      Use the calculation to check data by double clicking on a column or by the drag-and-drop method.
\n\n\n\n\n·      The other part of the use case is to verify the same calculation done in SME by creating a calculation in OAC.\n·      For that verification, right click on my calculations and select create calculation.
\n\n\n·      Just like in SME in OAC, we can use the in-built analytical expressions or type-in.\n·      But to select a column, we must hover to the desired fact/dimension and double click or drag-and-drop on the column.\n·      We select “other amount” from fact and “customer class” from account dimension and limit customer class to ‘Residential’.
\n\n
\n\n\n·      Give the name of calculation and click on validate to see for any errors>then click on save.
\n\n\n·      Verify the data of the calculation from My Calculations by double-clicking on it.\n·      We can verify that the SME calculation and the OAC calculation is the same.
\n\n\nNEED MORE GUIDANCE? CLICK HERE TO CONNECT WITH US.\n


FDI Use Case: OUDI Customization Flexibility

By Srivyshnavi Akarapu, HEXstream data analytics engineer\nIn this Tech Corner feature we will spotlight the third-party data connectors available in Oracle fusion data intelligence and demonstrate an example of how to connect to one.\nUnderstanding third-party data connectors\nThird-party data connectors are prebuilt or configurable integration points that allow Fusion Data Intelligence (FDI) to connect, extract and load data from external systems. They eliminate the need for building custom integrations from scratch, saving both time and cost.\nSome data connectors are available as preview features, such as:\nPreview features are disabled by default we can enable them by using enable features option. \n\n\n\nAWS S3, Azure SQL, Azure Storage Connector, Oracle Autonomous Database, Snowflake, SQL Server, Remote Agent, Oracle EPM-Enterprise Data Management, and many more.\nData connectors available by default\nGoogle Analytics: With the help of this feature, users can connect to a Google Analytics instance and use the data to create a comprehensive controlled-data pipeline.\nOracle Object Storage Service: This managed pipeline connects to Oracle Object Storage to ingest datasets into Oracle Fusion Data Intelligence followed by data augmentation.\nSalesforce: With the help of this capability, an administrator can establish a connection to a Salesforce SaaS source and use the data to create a comprehensive controlled-data pipeline.\nCreating Azure storage connection\nIn Oracle Fusion Data Intelligence, create the Azure Storage data connection using these steps:\n·      Select Data Configuration under Application Administration in Oracle Fusion Data Intelligence.\n·      Select Manage Connections on the Data Configuration page.\n·      Next click Create and then click Connection.\n·      In the Create Connection option, choose Data Extraction for Usage Type, and then select Azure Storage as the connection type.\n·      In the Azure Storage connection dialog, choose Standard under Connectivity Type, type an email address in Notification Email to get alerts, enter the Azure Storage details, and then click Save.\n·      Next select connectivity Type as Standard.\n·      Enter an email address to receive notifications.\n·      Select File Type as CSV.\n·      In Azure Blob Connection String Enter storage account's connection string.\n·      In container Specify the Azure container.\n·      Enter Remote Host Extract Files Directory: details, Name of the base folder in which you must place all your data files in Azure Storage.\n·      CSV Delimiter: Delimiter for the data files.\n·      CSV Date Type: Your Azure Storage Source’s date format and the data files' date format must match.\n·      Make sure the Refresh Metadata option is enabled to ensure that metadata is refreshed when we save the connection. Also, we can refresh the metadata later from the actions page of Manage Connections.\n·      Click Save.\n·      Test the connection for the Azure storage on the Manage connections. \nConclusion\nThird-party data connectors in Oracle Fusion Data Intelligence make it easy to integrate data from external sources such as Azure Storage. By creating an Azure Storage connection, organizations can seamlessly bring in data without complex coding or manual effort.\nAfter the connection is created, we can use the data-augmentation option to create datasets and extend the semantic model, or we can create our own and use that data to build reports.

\nCLICK HERE TO CONNECT WITH HEXSTREAM ABOUT INTEGRATING YOUR DATA.\n

Tech Corner Tutorial: Third Party Data Connectors In Oracle Fusion Data Intelligence

By Krishna Sai Geetla, HEXstream data analytics engineer\nLet's explore how to modify existing subject areas by extending a new column that is brought via third-party data through SME (extending the OOTB Dimension).\nGoal: To verify if an out-of-the-box entity can be extended with new columns with data that is brought via a third-party source. To verify if the changes are reflected in the OOTB subject area and validate the results in OAC reports.\nKey points to consider\n→ Third-party data source for this use case is considered as a dataset coming through an Excel sheet.\n→ There must be a custom schema created in target ADW database with its own user-login details. \n→ The new table must have matching primary key data to join with the out-of-the-box fact.\n→ Both the primary key names for existing fact and new dimension must not be same.\nDetailed steps\n•	Connect to the target database with custom-user credentials.\n•	To import the new data from the Excel sheet, right click on “Tables” and then select “Import Data”.\n\n\n\n\n•	Select the file with data. \n\n•	Select format as Excel 95-2003(xls) if it is not automatically and click “Next”.\n\n•	In next wizard, select import method as Insert.\n•	Give the table name and click “Next”. \n\n•	Choose the columns from the Excel sheet that need to be attributes in the new table in DB and click “Next”.\n\n•	Change datatypes and size as per requirement or according to the columns in existing fact.\n•	Both the primary key names for existing fact and new dimension must not be same.\n•	The data type of primary keys of the two joining entities( custom and existing) must be same.\n\n\n•	Click on finish in the import summary wizard.\n\n \n\n•	For the “OAX_USER” to read/edit the data in new custom dimension in custom schema, there must be grants on the new table. \n•	In OAC “OAX$OAC” must have read access to the custom table.\n•	Use the following grants:\nGRANT SELECT ON CUSTOM_OUDIDEV2. CUSTOM_PERSON_ENV_CLASS TO OAX$OAC;\nGRANT ALL ON CUSTOM_OUDIDEV2.CUSTOM_PERSON_ENV_CLASS TO OAX_USER;\n\n•	In the Admin console, go to Semantic Model Extensions.\n\n\n•	In user extensions select create sandbox and enter a name. \n\n•	Select “Manage logical star” in “perform action” dropdown. \n\n•	Select the edit logical star->out of the box->Subject area name-> Fact name and click on “Next”. \n\n•	Right click on the Required dimension for the columns to be added. Select “manage extension”. \n\n•	In the manage extension dropdown select extend dim.\n\n\n•	Select the Schema name and table name in object. Select the attributes that needs to be added in existing dimension from custom data and click on “Next”. \n\n\n•	Give appropriate join->select joining keys ->Click on Finish.\n\n •	Verify the new change and go back.\n\n •	Make changes in subject area by selecting manage SA from perform action dropdown.\n\n\n•	Verify Subject area folders and click “Next”. \n\n•	In 2nd Wizard click on manage elements dropdown and select Manage Factory Customizations.\n\n \n\n•	Select the columns by checking and click on add folders.\n\n •	Select the dimension folder to get the new changes there and click add folders.\n\n•	Verify the details of new added folder and click on “Finish”.\n\n•	Go back to user extensions and apply changes on sandbox.\n\n•	Check the status in activity and go to SME tab.\n\n•	Publish the new changes.\n\n \n\n•	After verifying publishing go to OAC page.\n\n•	Select the subject area in the data tab.\n\n•	Verify if new columns are added into existing dimension.\n\n \n\n•	Verify the data of new extended columns in report.\n\n\nNEED MORE GUIDANCE? CLICK HERE TO CONNECT WITH US. \n\n\n\n

Tech Corner Tutorial: OUDI Product-Customization Flexibility Through FDI

By Satish Saini, HEXstream utilities industry specialist\n \nLet’s talk electric vehicles, charging those EVs, and the infrastructure required to properly manage this increasing trend.  \nMost EV-charging infrastructure is connected to the utility grid, drawing power through grid assets. (The exception being stand-alone infrastructure). And any operational or performance issues with utility-grid assets directly affects EV-charging performance and efficiency. In a nutshell, when grids have problems, vehicles might not get charged.  \nThe functionality of EV chargers is fully influenced by upstream grid-asset capacity, the operating conditions of those grid assets, and operational-performance parameters. The same is true in the opposite sense, too. EV-charger infrastructure, operations and functionality also directly affect utility grid assets. \n\nSo what does this mean? For one, there is a growing need for accurately mapping EV-charging infrastructure with connected grid assets. We need to understand what is connected, where it is connected and what and how it is working, etc. We also need to properly integrate the operational and performance data of grid assets (such as distribution transformers, feeders and substations) with the functioning data from EV chargers. This includes both real-time and historical data, along with event logs. \nThis is no small feat, particularly as more EVs are purchased and more charging infrastructure stations complicate these challenges.  \nThe grid-asset data that is produced in this process—data that requires proper mapping, of course—includes:  \nAsset type \nAsset location / ID \nCapacity \nLoad patterns \nPhase unbalancing and fluctuations \nVoltage, sags and spikes \nFrequency, dips and fluctuations \nOutages and interruptions, including momentary interruptions \nOn the other hand, key EV-charger metrics that  are Influenced by utility-asset performance can be defined as: \nCharger location and mapping-to-grid assets \nCharger availability, utilization and uptime \nCharging sessions over time (daily, weekly, etc.) \nEnergy/kWh dispensed \nPower drawn, including maximum / peak power \nCharger-session failure rate \nCharger efficiency \nFault rate \nCustomer experience \nSmarter grid-asset management=better grid-investment planning  \n \nUtilities that have effective asset-management strategies reliant on health / performance insights gleaned via operational-data analytics can approach EV infrastructure much more wisely. These utilities can plan, prioritize, install, operate and manage EV chargers with enhanced efficiency and lower charger-downtime. They can similarly manage their grid assets more effectively while providing key decision-making resources for grid-investment planning.  \nSimple? Not necessarily. But certainly achievable with the right commitment and solution provider. Some of the key elements to consider related to data integration / mapping with EV-charging assets include:  \nUtility-grid schematics and single-line diagrams, along with major assets and corresponding EV chargers’ location maps. \nKey sources and platforms for data on grid assets, EV chargers and vehicles (including  assets registry, assets operating data platforms, AMI / metering data, EV chargers’ data as defined above, third party / external data platforms for vehicle and chargers’ data).  \nData-acquisition from all sources, along with the mode of communication, including both real time and historical data. \nAdvanced analytics and AI / ML applications for assets-loading (mapped with EV chargers’ operation), assets behavior and charging patterns, overloading / predictive maintenance and failure forecasts, optimization algorithms for grid assets and EV charger infrastructure, opportunities for demand response, tiered pricing, peak-load management and vehicle-to-grid energy. \nAlerts and notifications on key issues including failure predictions and overloading  for effective decision-making with assets and chargers’ operation, including short-term and long-term grid-investment planning. \n\nWANT MORE? CLICK HERE TO CONTACT US ABOUT MAPPING YOUR GRID-ASSET DATA WITH YOUR EV-CHARGING INFRASTRUCTURE.\n  

Grid-Asset Data Grows Even More Complex (And Even More Valuable) With EV Chargers In The Mix

We frequently hear about DERMS in the utility space, but some of us don't have a full grasp on the capabilities and wins related to Distributed Energy Resource Management Systems. Here we chat with our Swathi Ambati, HEXstream data solutions engineer, to address those issues. Take a look...\nHEXstream: What is DERMS?\nSwathi: A Distributed Energy Resource Management System (DERMS) is an advanced software platform that enables electric utilities to monitor, control, optimize and orchestrate distributed energy resources (DERs)—such as rooftop solar, battery energy storage, electric vehicles (EVs), smart thermostats, and controllable load devices—at both the grid edge and system-wide levels.\nIt provides situational awareness and automated control to maintain grid reliability, support load-balancing, and participate in wholesale and retail market programs.\nHEXstream: Who is DERMS built for?\nSwathi: Let's consider the DERMS project currently in progress; Oracle and HEXstream are partnering on an implementation for Nevada Energy. HEXstream is focused on custom report development with the goal of modernizing their demand-response programs and distributed-energy resource operations.\nHEXstream: What problems does DERMS solve, and what benefits does it provide?\nSwathi: The solution offers a wide range of functionalities, including:\nCreating and managing demand-response programs\nMigrating data from existing legacy systems\nHandling new program registrations\nSupporting customer inquiries related to DER programs\nManaging CSR (Customer Service Representative) and self-service workflows\nCoordinating service requests and appointment scheduling\nIntegrating with third-party field service providers\nTracking and managing field device inventory and lifecycles\nEnforcing program rules, device capabilities and contracts\nRecording event and device-participation histories\nSupplying per-device control scores to the Network Management System (NMS) for day-ahead scheduling\nProviding settlement data for billing purposes\nHEXstream: How is this DERMS different from other solutions?\nSwathi: The DERMS solution leverages DACS, which acts as the core engine to create, manage and optimize demand response and distributed-energy programs. Unlike other solutions, it offers truly integrated and end-to-end DER management experience, driving both operational efficiency and compliance.\nHEXstream: What environmental, regulatory, financial, and customer factors are influencing DERMS adoption?\nSwathi: Several evolving conditions make DERMS more attractive to utilities:\nEnvironmental: Growing climate concerns and renewable-energy targets\nRegulatory: Compliance with clean-energy mandates and grid-reliability standards\nFinancial: Incentives for DER participation and operational cost savings\nCustomer Expectations: Demand for energy choice, transparency and control\nElectric Vehicles (EVs): Increasing EV adoption that necessitates grid flexibility and localized energy management\nHEXstream: What reporting capabilities are built into the DERMS solution?\nSwathi: Based on Nevada Energy's requirements, the reporting framework is structured around three key areas—Program Management, Field Service Organization, and NMS Forecast. Key capabilities include:\nDashboards and reports to track:\nNumber of customers and devices enrolled\nAttrition rates\nProgram performance over time\nData visualizations to assist with customer targeting and marketing\nHistorical and forecast data for individual DER participants\nPre-built field service operation reports\nAsset-search functionality by parameters like device type \nHEXstream: How might DERMS evolve over the next decade?\nSwathi: In the next 10 years, DERMS will likely evolve to support:\nReal-time grid optimization using AI/ML\nFull-scale integration with smart home ecosystems\nAutonomous grid response to extreme weather or high demand\nPersonalized energy insights for consumers\nAdvanced analytics and predictive forecasting\nSeamless participation in energy markets by individual prosumers\n\nWANT MORE? CLICK HERE TO CONNECT WITH US TO EXPLORE HOW TO OPTIMIZE DERMS AT YOUR UTILITY.\n \n


The Distributed Energy Resource Management System: Optimizing DERMS At Your Utility

By Shanthan Aluguvelli, AI Nexus data solutions engineer\nIn today's digital-first world, organizations are increasingly turning to advanced technologies to enhance data accessibility while optimizing operational efficiency. Among these innovations, knowledge-based chatbots stand out as transformative tools that are reshaping how teams interact with complex information systems. \nAt HEXstream, this newly developed specialized chatbot solution represents the cutting edge of this technology, designed specifically for the unique challenges of data-intensive industries.\nThis chatbot is available as a web widget within \nHEXpert\n, the AI-based knowledge-management platform. This chatbot is a built-in feature of HEXpert, whereas it’s not available out-of-the-box in ChatGPT or Co-Pilot. Additionally, this chatbot widget is flexible—it can be implemented independently on a client’s platform or instance, depending on their specific needs, even if the utility chooses not to use the full HEXpert solution.\nUnderstanding knowledge-based chatbots\nKnowledge-based chatbots differ from traditional rule-based conversational agents by leveraging sophisticated AI to access, interpret and deliver information from a structured knowledge base. Rather than following pre-programmed response paths, these intelligent systems can:\nProcess natural-language queries with contextual understanding\nAccess comprehensive databases of organization-specific information\nDeliver precise, relevant answers tailored to user inquiries\nLearn and improve from each interaction\nScale to handle thousands of simultaneous conversations\nThe HEXstream chatbot is built on a sophisticated architecture that combines several cutting-edge technologies. At its core, the system utilizes a large language model fine-tuned specifically for technical terminology and concepts, enhanced with retrieval-augmented generation techniques. This enables the chatbot to pull from organizational documentation, knowledge bases, and historical patterns stored in a vector database for efficient retrieval based on conversational context.\nThe system includes a conversation-management layer that maintains dialogue history, tracks user intent, and manages multi-turn interactions about complex topics. Rather than starting with an empty knowledge base, the chatbot undergoes a comprehensive training process using organization-specific documentation and logs, supplemented with industry standards and best practices.\nIntelligent interaction capabilities\nConversational-knowledge interface\n—The HEXstream chatbot provides an intuitive conversational interface to organizational knowledge, allowing users to ask questions in natural language. When someone inquires about specific information, processes or definitions, the chatbot retrieves relevant information from its knowledge base. It then presents results in a conversational format, explaining what it found and offering to refine the search based on follow-up questions.\nFor example, when a team member asks, "What's our process for handling customer escalations?" the chatbot identifies relevant documentation, extracts the key information, and returns meaningful insights rather than raw data. This natural-language processing capability makes knowledge access seamless for everyone, regardless of their technical proficiency.\nSeamless multi-channel deployment\n—This knowledge-based chatbot is designed to meet users wherever they are, providing consistent information across multiple communication channels:\nWeb portals and mobile applications\nSMS text messaging\nSocial-media platforms\nVoice interfaces (phone systems)\nSmart home devices\nIntegration with collaboration platforms like Microsoft Teams and Slack has proven particularly valuable, enabling teams to access knowledge without leaving their primary work environments. This seamless integration reduces context switching and enhances productivity by bringing information directly into existing workflows.\nReal-world applications\nStreamlining daily operations\n—The chatbot has transformed routine operations for many organizations. Team members use it to quickly check policies or procedures, and reference information through natural conversation. Technical staff leverage the chatbot during troubleshooting to access documentation and best practices without context switching between multiple knowledge bases.\nBusiness analysts who previously relied on specialized teams for information access now independently explore organizational knowledge through conversational interactions. This self-service capability has significantly reduced the workload on subject-matter experts while accelerating insight generation across organizations.\nEnhancing outage communication\n—During critical system outages, the chatbot serves as a central communication hub, providing consistent, accurate information to stakeholders across the organization. Support teams can quickly ask, "What's the current status of the network outage?" or "Which services are affected by the current incident?" and receive up-to-date information without accessing multiple monitoring systems.\nThe chatbot's value extends to business-continuity planning, where teams can ask, "What's our protocol for this type of outage?" and receive clear guidance based on established procedures. This capability ensures that all stakeholders have a shared understanding of the situation, reducing confusion and enabling better decision-making during critical incidents.\nWhen service disruptions occur, the chatbot enables non-technical stakeholders to get clear, jargon-free updates. Business leaders can ask, "How is the current outage affecting customer transactions?" and receive comprehensible explanations without technical teams needing to prepare special reports. This transparency helps maintain trust and alignment during challenging situations.\nFuture directions\nWe continue to enhance this chatbot with more sophisticated dialogue capabilities and deeper integration with enterprise systems. Future versions will support more complex reasoning about organizational knowledge and provide proactive suggestions based on usage analysis. We're also exploring integration with workflow-automation tools, enabling the chatbot to not just answer questions but also initiate actions based on conversational requests.\nThe knowledge-based chatbot represents a fundamental shift in how organizations interact with their information systems. By combining advanced AI techniques with specialized knowledge management, we've created a conversational interface that makes information more accessible and actionable for everyone. \nFrom daily operations to critical-outage scenarios, the HEXstream chatbot serves as an intelligent assistant that enhances productivity, improves communication, and democratizes access to valuable organizational knowledge across teams.\n\nWANT MORE? CLICK HERE TO CONNECT WITH US ABOUT ENHANCING YOUR DATABASE INTELLIGENCE WITH THE HEXSTREAM CHATBOT\n\n.\n \n


Introducing A Smarter Chatbot For Data-Intensive Industries

By Satish Saini, HEXstream utility industry specialist \nAs we explored in \nprevious features\n, utilities need effective asset-management strategies, programs and performance insights with relevant analytics in response to:\nAging T&D grid infrastructure leading to frequent and untimely equipment failures\nClimate change causing adverse weather events and undermining power-infrastructure reliability\nIncreasing customer demand for reliable power\nUtilities’ needs for sustainable, efficient and cost-effective business operations\nRegulatory pressure to maintain operational and safety standards\nRenewable-energy sources penetration to the T&D grid\nA key component in developing these smarter asset-management strategies is asset performance insights (API), which involves unique methodology and metrics using AI/ML algorithms paired with deep utility operational and functional expertise to monitor and evaluate the overall health/risk index for various critical grid assets and equipment. API optimizes equipment operation and condition-based maintenance, improving reliability and lifecycles while reducing maintenance costs.\nThe benefits \nEffective asset-management plans and automated performance insights using AI/ML algorithms to generate API can help:\nDefer replacement and upgrade costs through predictive analysis regarding assets’ health condition\nEnable critical condition-based monitoring to avoid major failures\nReduce costs for maintenance of critical and expensive assets such as transformers and circuit-breakers\nMaximize the lifecycle of assets\nKey components and processes\nAPI uses digital-transformation tactics and real-time monitoring of assets, extracting and analyzing historical, operational and performance data from multiple sources and platforms for data-driven analytics and decision-making metrics/KPIs. \nThis methodology enables the creation of notifications regarding asset health/wellness, and a risk index based on a defined scoring pattern. \nHEXstream is developing an API analytics tool with key indicators on various metrics for major assets across the T&D grid. In Phase 1 of this project, our focus is on power transformers installed at transmission and distribution stations.\n\n\nIn future phases, we will include other assets like circuit-breakers, reclosers, etc. We will explore major indicators, sub-indicators and metrics along with different layers of information for calculating the risk/wellness index for effective utility-asset management.\n\nWANT MORE? CLICK HERE TO CONTACT US TO LEARN HOW API CAN HELP YOUR UTILITY MANAGE CRITICAL ASSETS. \n\n


Developing Smarter Asset Strategies From Deeper Asset Performance Insights

By Bharath Moola, HEXstream data solutions engineer\nEvery time you flip a switch, flush a toilet, or glance at a smart meter, you rely on a massive, complex network of pipes, transformers, substations, meters and sensors. Utility companies manage this vast system to keep services running smoothly, but the infrastructure behind it all is more complicated than most people realize. \nUnfortunately, many utilities still use traditional databases that organize data into flat tables. The problem is that utility infrastructure is not simple or linear—it’s a web of connections, constantly changing and interacting. \nNeo4j offers a new way to handle this complexity by focusing on how things connect.\nWhy traditional databases fall short\nTraditional databases store information in tables and depend on joining these tables together to represent relationships. This method works for simple data but breaks down when relationships become complex or change often. Utility networks aren’t just a collection of separate parts; they form an interwoven system where the state of one component affects many others. \nTrying to represent these dependencies with traditional databases results in slow queries, complicated schemas, and inflexible systems. Delays in understanding these connections can mean slower responses and longer outages, something utility companies cannot afford.\nNeo4j: Thinking in connections, not tables\nNeo4j approaches data with a different mindset. Instead of tables, it uses nodes to represent entities such as meters or transformers, and relationships to show how these entities link. This graph-based model better matches the real world and enables fast, natural exploration of data. Queries that might take minutes or even hours with relational databases happen in seconds with Neo4j because it navigates connections directly. This means utility companies can model their entire infrastructure as it truly exists and answer complex questions on the fly.\nAll-in-one intelligence: From faults to forecasts\nWith Neo4j, utility companies unlock a wide array of capabilities all within one system. When a component fails, the graph reveals exactly which parts of the network are affected. Maintenance teams can trace asset dependencies to plan repairs that minimize disruption. Real-time data from smart meters and sensors can be linked directly to infrastructure, helping spot anomalies or optimize power distribution. \nAuditors gain clear insight into the sequence of events before and after incidents. Customer service improves as customer profiles connect with service areas and outage information. Even “what-if” scenarios become easy to explore—like predicting the impact of taking a pipeline offline. Neo4j’s flexible design allows all these uses to coexist, adapting as new technologies and needs arise.\nFrom data storage to data insight\nWhat makes Neo4j truly powerful is how it turns raw data into a living map of the utility network. Instead of drowning in disconnected data, utility teams gain a clear picture of how everything relates. This clarity leads to smarter decisions and faster responses. The database grows with the system, letting utilities stay agile in a world that never stops changing. In industries where connections matter most, Neo4j doesn’t just store data—it brings it to life.\nFinal thoughts\nUtility networks are more complex than ever before, and traditional ways of managing data no longer work. Databases that treat information as isolated tables miss the bigger picture of how everything fits together. Neo4j changes that by focusing on connections, enabling utilities to see their systems as they truly are. This shift helps teams respond faster to outages, plan maintenance better, and make sense of data from smart devices in real time. \nMost importantly, it turns mountains of raw data into clear insight, helping utilities act with confidence and speed. The future of utility management depends on understanding these connections—and Neo4j makes that future possible today.\n\nCLICK HERE TO CONNECT WITH US TO LEARN MORE ABOUT NEO4J.\n \n


Rewiring Utility Intelligence: How Neo4j Changes The Game

By Bala Sai Tilak.Thalla, HEXstream data integrations engineer\nOracle Integration Cloud (OIC) is a powerful cloud-based PaaS platform designed to integrate heterogeneous and homogeneous applications seamlessly. Known for its ease of use, OIC supports a wide variety of systems via prebuilt adapters, making it a go-to solution for enterprise integration.\nTraditionally, OIC receives updates on a quarterly basis. However, in August 2022, Oracle introduced a major upgrade with the release of ‘OIC Gen 3’, a new generation of the platform packed with advanced capabilities and a modern interface. \nIn this blog, we will explore the standout features of OIC Gen 3 and how it empowers developers to build robust and scalable integrations.\nA fresh new look: redwood-inspired interface\nOne of the most noticeable differences in OIC Gen 3 is the user interface. While Gen 2 featured a blue-themed design, OIC Gen 3 introduces a Redwood-inspired UI, aligning with Oracle’s latest design system. This cleaner, more modern look enhances the overall user experience and makes navigation more intuitive.\n\n\nIntroducing projects to organize Integrations\nOIC Gen 3 introduces a concept called Projects, which act like workspaces that house related integration assets in a structured manner.\n\n\nEach project can contain:\nUp to 100 Integrations\n50 Deployments\n50 Connections\n50 Lookups\n20 JavaScript Libraries\n\n\n\nProject tabs:\nDesign: Where integrations are built\nDeploy: Deploy multiple integrations together\nObservability: Monitor integrations by instance, schedule, or audit trail\nProjects also support role-based access control, allowing users to edit, view, or monitor components based on their roles.\nExtended retention period\nIn OIC Gen 2, the retention period for viewing integration instances (success/faulted) was three days by default, configurable up to 30 days.\nIn OIC Gen 3, this limit has been increased to 32 days, allowing developers and support teams to track integration histories for longer durations.\nTracing levels: debug, audit and production\nOIC Gen 3 introduces three levels of tracing to balance visibility and performance:\n\nTracing Level\n     Purpose\nDebug   \nCaptures full payloads for detailed testing and debugging\nAudit\nRecords trigger/invoke input-output payloads with moderate detail\nProduction\nOptimized for performance with minimal logging to ensure security and speed\nThese options offer better control over how much runtime data is stored and when.                  \nIncreased payload sizes\nA major improvement in Gen 3 is the increase in payload size limits for key adapters:\nAdapter   \nGen 2 Limit
   \nGen 3 Limit\nREST\n10 MB\n   100 MB\nSOAP\n10 MB\n   100 MB\nSimilarly increased the payload limits for the database, FTP and other adapters for both trigger and invokr levels. This enhancement enables handling of large files and complex data structures with ease.\nEnhanced actions and new sections\n\nOIC Gen 3 revamps the action palette by introducing new action categories and capabilities, including:\nOCI AI services section\nDedicated to Oracle AI integrations such as:\nOCI Generative AI\nOCI Vision\nOCI Speech\nOCI Documentation AI\nOCI RAG (Retrieval Augmented Generation)\n\nThese allow you to build AI-powered integrations directly within OIC.\nParallel action: Found under the Logic section, this new action enables parallel processing, allowing
multiple endpoints to be called simultaneously—improving performance and response times.\n Increased iteration capabilities\nFor-each Loop: Increased from 5,000 iterations to unlimited\nWhile loop: Remains at 5,000 iterations\nThis change enables handling of large datasets without artificial loop limits.\nAdvanced mapping capabilities\nOIC Gen 3 enhances the Mapping editor with a new Import Mapping feature. \n\n\n\nYou can now:\nImport .XSL files directly into your mappings\nReuse complex transformation logic across multiple integrations\nStandardize mapping logic across similar scenarios\nRicher function library for mappings: Mapping transformations have been significantly improved with a broader set of built-in functions, making it easier to manage complex logic. This reduces the need for external scripting or additional transformation steps.\nIncreased integration limits\nOracle has raised the active integration count from 700 to 800, supporting organizations with larger-scale integration requirements.\nOracle Integration Cloud Gen 3 is not just a cosmetic upgrade, it’s a major architectural and functional enhancement. By leveraging the new features in OIC Gen 3, integration developers can build lightweight, high-performing, and future-ready integrations for any enterprise need.\n\nNEED HELP WITH OIC? CLICK HERE TO CONTACT US. 
\n

Exploring The Advanced Features Of Oracle Integration Cloud (OIC) Gen 3

By Nayana Kandalam, HEXstream data analyst\nIf you've ever worked with Power BI, you know it can be an absolute lifesaver for turning data into insights. But let’s be real—nothing is more frustrating than a slow, clunky dashboard that takes forever to load.\nHere’s how you can supercharge your Power BI experience with just a few simple adjustments. These tweaks will help make your reports faster, reduce manual tasks, and make your Power BI usage smoother overall.\n1. Build a clean and efficient data model\nThink of your data model as the backbone of your Power BI reports. If the model is messy, everything else will be, too. Here are some quick ways to clean it up:\nStar schema is your friend: Instead of cramming everything into a single table, organize your data into separate fact and dimension tables. This structure enhances performance.\nImport mode is faster: When you import data into memory, your reports will load much more quickerly. Only use DirectQuery if you need real-time data updates.\nRemove unnecessary data: Avoid loading unnecessary columns and rows. Less data means faster refresh times.\nPre-aggregate data: Instead of using raw transaction-level data, aggregate your data into daily, weekly, or monthly summaries to cut down processing time.\n2. Optimize your DAX formulas for speed\nDAX is an incredibly powerful tool in Power BI, but inefficient DAX formulas can slow things down. Here’s how to streamline them:\nUse measures instead of calculated columns: Measures are dynamic and don’t take up space in your data model, making them more efficient than calculated columns.\nSwitch out nested IFs for SWITCH: The SWITCH function is cleaner and processes faster than nested IFs.\nAvoid DISTINCTCOUNT when possible: DISTINCTCOUNT can be resource-heavy. Try using SUMX or aggregating your data beforehand for better performance.\nLeverage variables: Using variables in your DAX formulas helps store intermediate results, which avoids recalculating values multiple times and improves performance.\n3. Automate tedious tasks & save time\nPower BI isn't just for creating beautiful reports; it can help automate repetitive tasks too:\nSet up automatic data refreshes: No need to manually refresh your reports! Schedule automatic updates in Power BI Service.\nUse Power Automate: Set up triggers and workflows with Power Automate. You can automate alerts (e.g., when sales drop below a certain threshold) or approval processes.\nDynamic parameters are key: Instead of creating multiple versions of the same report, use dynamic filters that let users adjust the report based on their needs.\nImplement row-level security (RLS): Keep your data secure and personalized without the need to create separate reports for each user.\n4. Streamline your Power Query transformations\nPower Query is where the magic happens during data transformations. Keeping it efficient is essential for speed:\nTurn off load for unused queries: If a query is just there for intermediate steps, make sure it doesn’t load into your model; this prevents unnecessary processing.\nCombine transformation steps: Minimize the number of steps by combining transformations. Each step adds processing time, so fewer steps means faster performance.\nFilter early: Instead of importing a massive dataset and filtering it within Power BI, apply filters directly at the source to reduce the data you’re working with.\n5. Make your dashboards load instantly\nA slow dashboard can lead to user frustration. Here’s how to ensure your reports load quickly:\nLimit visuals on a page: Each visual in Power BI runs a separate query, so fewer visuals means faster load times. Focus on the most critical ones.\nSummarize data: Instead of pulling in millions of rows of data, aggregate it. Summarized data loads much faster than raw data.\nUse hierarchical or dropdown slicers: Slicers with too many unique values can slow things down. Hierarchy slicers or dropdowns reduce this issue.\nUtilize the performance analyzer: This built-in tool helps you identify slow visuals or queries, so you can optimize them for better performance.\n6. What’s new in Power BI? (And why you should care)\nPower BI is always improving. Staying up-to-date means you can take advantage of the latest features to enhance your experience:\nAI & machine learning: Power BI now has built-in AI features like anomaly detection and predictive analytics, which help you spot trends without coding.\nSeamless Power Platform integration: Power BI works seamlessly with Power Apps and Power Automate, so you can create more interactive and automated solutions.\nCloud & hybrid connectivity: Power BI now supports multi-cloud environments like Azure, AWS, and Google BigQuery, making it easy to connect to different data sources.\nMobile optimization: Power BI’s mobile experience has improved, ensuring that your reports adjust perfectly to mobile screens.\nEmbedded analytics: You can now embed Power BI reports directly into apps or websites, bringing data to users wherever they are.\nImproved security: Features like Purview integration, sensitivity labels, and data-loss prevention (DLP) ensure that your data remains safe and compliant.\nFinal thoughts: Power BI is a superpower (when optimized right!)\nPower BI is more than just a reporting tool. When optimized, it can:\nMake your reports load faster\nAutomate repetitive tasks, saving you time\nCreate dashboards that are both insightful and user-friendly\nApply these tips to your workflow, and you’ll experience the true power of Power BI!\n\nWANT MORE? CLICK HERE TO CONNECT WITH US ABOUT OPTIMIZING YOUR POWER BI INITIATIVES.\n

Power BI Performance Hacks: Make Your Reports Faster & Smarter

HEXpert vs. The Competition

\nBy Ramaswamy Iyappan, HEXstream SDE\nAI agents are rapidly evolving from basic prompt-response tools into systems that can reason, use tools, and operate in complex, real-world environments. But there's a hidden superpower behind the most capable agents today—and it’s not just the language model (LLM) itself. It’s something deeper, more structured: the ReAct pattern.\n\n\nThis post breaks down what ReAct is, why it matters, what problems it solves, and how new frameworks like LangGraph are making it radically easier to build scalable, production-grade AI agents.\nWhat's an AI agent, really?\nLet’s start with the basics. At its core, an AI agent is a system that can:\n●      Perceive (observe the environment)\n●      Reason (make decisions based on observations)\n●     Act (take steps toward a goal)

Most AI agents today are powered by LLMs (like GPT-4 or Claude), but just using an LLM isn’t enough. Raw LLMs are stateless and reactive—they respond to a prompt, but they don’t have memory, structured reasoning, or the ability to call tools and APIs in a controlled way. That’s where agent frameworks come in.\nLLMs are not agents\nLet’s be clear: an LLM is not an agent. It’s a powerful foundation—but without structure, it can hallucinate, lose track of multi-step goals, or fail to make rational tool calls. Agents solve this by orchestrating LLMs with memory, tools, environment feedback, and multi-step workflows. This is where the ReAct pattern comes in.\nWhat is ReAct? \nReAct stands for Reasoning + Acting. It’s a pattern that lets an LLM "think before it acts," mimicking how humans solve complex tasks. Here’s how it works in practice:\nThe agent reasons about what it should do.\nBased on that reasoning, it selects and executes a tool (like a web search, code execution, database query).\nIt observes the result and loops back—continuing the reasoning-acting cycle until the task is complete.\nInstead of treating the LLM like a black box that returns final answers, ReAct exposes the intermediate thought process—and integrates external tools to ground that thinking in reality.\nWhy ReAct matters\nWithout ReAct-style logic, you run into three major issues:\nLLMs are prone to confidently making up facts. ReAct reduces hallucinations by grounding decisions in real tools and APIs.\nMost real-world problems aren’t solved in one step. ReAct supports chain-of-thought reasoning, allowing the model to break down tasks and iterate intelligently.\nTool calls without context often fail or return irrelevant results. ReAct lets the agent reason about what tool to use and why, improving accuracy.\nIn short, ReAct makes agents more intelligent, reliable, and transparent.\nHow LangChain implements ReAct \nLangChain was one of the first frameworks to formalize agent behavior using ReAct. It lets you build agents where:\n●     LLMs generate reasoning + tool selection\n●     The system executes tools\n●     The process loops until a final answer is reached
\nLangChain’s agents have modular components:\n●      LLMChain: to generate thoughts and actions\n●      Tool Executor: to run tools like web search or code execution\n●      Memory: to keep track of past steps\n●     CallbackManager: for step-by-step tracing and debugging\nBut as powerful as this is, it gets complicated fast—especially for non-trivial workflows involving
branching, retries, or multiple paths of execution.\nEnter LangGraph: ReAct at scale \nLangGraph takes the ReAct pattern and makes it production-grade. Instead of chaining
steps in a linear or looped fashion, you build a graph where each node is:\n●     A reasoning step\n●     A tool call\n●     Or any async function\nAnd the edges define how the agent moves based on state and observations.\nWhy LangGraph is a game-changer\n●      Graph-based logic: Think workflows, not chains. Nodes can branch, loop, retry, or short-circuit.\n●      State management: Each node sees and updates the agent’s state, making the system fully reactive.\n●      Durability: Supports retries, checkpoints, and memory out of the box.\n●     Developer-friendly: Built with LangChain-native abstractions, plus integration with LangSmith for debugging and tracing.\nKey benefits of LangGraph for ReAct-style agents\n✅ ReAct logic is built in—you don't reinvent loops and reasoning.\n✅ Async-native—ideal for real-time applications.\n✅ Retry + persistence—no more brittle agents that fail on one bad call.\n✅ Built-in memory and tracing—crucial for debugging and improving agent behavior.\n✅ Open-source—transparent, extensible, and community-driven.\nWhat this means for AI developers in 2025\nIf you’re building AI workflows or autonomous systems, here’s the new blueprint:\nChain-of-thought + structured tool use is the gold standard for reliability.\nWorkflow graphs mirror real-world logic better than linear chains.\nYou’ll ship faster, debug easier, and scale cleaner—without callback hell or custom scaffolding.\nFinal take\nReAct isn’t just a clever acronym. It’s the architecture that’s powering the next generation of intelligent, grounded, reliable AI agents.\nAnd LangGraph? It just made that architecture modular, maintainable, and production-ready.\nIf you’re building AI agents that actually work in the real world, it’s time to move from chain-based hacks to graph-based systems.
\n\nWANT TO LEARN HOW THIS COULD WORK AT YOUR ENTERPRISE? CLICK HERE TO CONTACT US. \n

The Hidden Superpower Behind Modern AI Agents: The ReAct Pattern (And Why LangGraph Changes Everything)

By Deepika Geetla, HEXstream Data Engineer\nWhen users interact withthe Oracle Analytics AI Assistant, a multi-step process takes place behind the scenario. In just a few seconds, the assistant translates a natural language question into a meaningful visualization—giving professionals fast, accurate and contextual insights.\n\n\nHow AI turns questions into visual insights\nLet us say, “What was the peak electricity demand in Texas last August?”\n1.    User Question Received: Oracle Analytics captures your natural language query.\n2.    Dataset Identification: Oracle Analytics identifies the most appropriate dataset based on your indexed metadata and recognizes relevant attribute matches.\n3.    Prompt Construction: Oracle Analytics prepares a comprehensive prompt for the Large Language Model (LLM) that includes dataset metadata, column names, synonyms you have defined, and the user’s original question.\n4.    LLM Processing: The language model evaluates the prompt and translates the question into a structured query with appropriate refinements.\n5.     Query Verification & Execution: Oracle Analytics validates the query against your data
model and executes it against your database.\n6.    Visualization Generation: Based on the query results and data composition, Oracle Analytics produces an appropriate visualization.\nLet us decode the AI into action\nImagine a customer service manager asks, “How many new electric connections were added in California last month?”\nOracle Analytics AI Assistant would:\n·      Map “new electric connections” to the “New Service Activations” metric and apply a COUNT or SUM aggregation as needed.\n·      Recognize “California” as a value in the State field.\n·      Interpret “last month” as a dynamic filter based on the current date, applied to the Connection Date or Activation Month dimension.\n·      Generate a bar or KPI visualization showing the total number of new connections for California for the selected period.\nWhy tuning makes or breaks your AI experience\nWhile the Oracle Analytics AI Assistant is powerful, its effectiveness depends on how well you prepare your data and metadata. As Oracle notes, “Gen AI is Magical, but It Is Not Magic.” The performance, accuracy and user satisfaction all hinge on proper tuning.\nThree areas where tuning drives success\n1. Accuracy amplifier: Getting the right answers, not just any answers\nWithout proper tuning: The AI Assistant may misinterpret questions or select incorrect fields. For example, if your dataset has similar columns like “Product ID” and “Product Name,” a question about “product sales” might confuse the AI about which field to use.\nWith proper tuning: When metadata is prepared with clear field names and synonyms, the AI Assistant correctly maps user questions to the right data elements. A question like “Show income by product” will accurately identify “income” as referring to your “Net Profit” measure.\n2. Speed booster\nWithout proper tuning: Indexing everything creates overhead and slower processing. The LLM must navigate irrelevant field names and values, which slows down interpretation.\nWith proper tuning: A focused index enables quicker processing. The first answer is more likely correct, so the overall time to insight is significantly reduced.\n3. Trust builder\nWithout proper tuning: When the AI Assistant frequently misunderstands questions, users become frustrated and lose trust.\nWith proper tuning: Users gain confidence when the AI Assistant consistently returns correct answers. This positive experience encourages continued use and wider adoption.\nThe Fantastic Four: Principles for AI analytics excellence\nFor ongoing optimization of your AI Assistant implementation, follow these key principles:\n1. Iterate: Continuously tune your metadata based on user feedback and observed usage patterns. Monitor which questions succeed or fail and refine accordingly.\n2. Describe: Ensure your metadata clearly describes your data in business terms that users
naturally employ, not technical database terminology.\n3. Clarify: Use synonyms to resolve ambiguity, especially for similarly named fields:\nMonth → Birth Month\nMonth → Hire Month\nMonth → Role Change Month\n4. Less is more: Be selective in what you expose to the AI Assistant. A focused, well-curated dataset will yield better results than including everything.\nThe Oracle Analytics AI Assistant works out of the box—but tuning your metadata makes a significant difference. It improves query accuracy, speeds up analysis, and builds trust across your analytics users.\nBy investing just a little time in indexing the right fields, renaming technical labels, and adding thoughtful synonyms, you can transform how your teams engage with data. What once required dashboards, filters and manual SQL can now be accomplished with a single question.\nWhen done right, Oracle’s AI Assistant bridges the gap between complex utility datasets and the way people naturally think and ask questions—turning data into conversations, and conversations into actionable insights.\n\nWANT MORE? CLICK HERE TO CONNECT WITH US TO SUPERCHARGE YOUR DATA.\n

Smart Insights Simplified: Oracle AI’s Data Discovery Flow

By Nayana Kandalam, HEXstream data analyst\nIn today's data-driven world, businesses rely on analytics to make informed decisions,
optimize operations, and gain a competitive edge. Microsoft Power BI has emerged as a leading business intelligence tool that simplifies data analysis, enhances reporting, and integrates seamlessly with various platforms to automate workflows. \nThis article highlights the critical features in Power BI that drive business success and how it integrates with the broader Power Platform to streamline processes.\nKey Power BI features that drive business success\n1. Advanced data analysis with AI-powered Q&A—Power BI’s Natural Language Processing (NLP) capabilities enable users to interact with data intuitively. With the Q&A feature, business users can ask questions in natural language and receive instant insights, eliminating the need for complex queries or technical expertise.\n2. Row-Level Security (RLS) for secure data access—Data security is paramount for business growth, and Power BI’s Row-Level Security (RLS) ensures that users only access the data relevant to their role. By implementing RLS, organizations can prevent unauthorized access, maintain compliance, and enhance data governance.\n3. Real-time data visualization and dashboards—Power BI allows businesses to create real-time dashboards that provide up-to-date insights. With live connections to various data sources, decision-makers can monitor key performance indicators (KPIs) and respond swiftly to changing business conditions.\n4. Integration with Power Platform for Automation—Power BI seamlessly integrates with Power Automate, Power Apps, and Power Virtual Agents, unlocking automation possibilities:\nPower Automate: Automate data extraction, report scheduling, and alerts based on predefined triggers.\nPower Apps: Create custom applications that interact with Power BI reports for data input and decision-making.\nPower Virtual Agents: Build AI-driven chatbots to answer business-related queries using Power BI insights.\n5. AI and machine-learning capabilities—With AI Insights, Power BI enables users to apply machine learning models, perform sentiment analysis, and detect anomalies without requiring deep technical expertise. These advanced analytics capabilities help businesses uncover hidden patterns in their data.\n6. Mobile accessibility and report sharing—Power BI offers a robust mobile app that allows users to access reports and dashboards on the go. Additionally, businesses can securely share insights through Power BI Service, ensuring collaboration across departments.\n7. Self-service and automated reporting—Manual reporting is time-consuming and prone to errors. Power BI simplifies this process through:\nAutomated data refresh to ensure reports always display the latest information\nDrag-and-drop visualizations for quick insights without the need for extensive training\nTemplates and pre-built connectors to streamline report creation and reduce manual effort\nThe impact of Power BI on business growth\nPower BI not only enables businesses to extract actionable insights but also enhances efficiency by integrating with automation tools, ensuring data security, and leveraging AI for intelligent decision-making. \nOrganizations that leverage Power BI can:\nMake faster, data-driven decisions\nImprove operational efficiency by reducing manual efforts\nEnsure data security and compliance\nScale analytics capabilities with AI-driven insights and automation\nA game-changer \nPower BI is a game-changer for businesses aiming to harness data for success. Its ability to integrate with the Power Platform, provide advanced analytics, and ensure secure data access makes it a vital tool for any organization. By leveraging Power BI’s capabilities, businesses can simplify manual efforts, enhance decision-making, and drive sustainable growth.\n\nARE YOU READY TO TRANSFORM YOUR PROCESSES WITH POWER BI? CLICK HERE TO CONTACT US TO LEARN HOW. \n


Unlocking Business Success With Power BI: Key Features & Integrations

By Ashwini Nagendra Prasad, HEXstream Integrations Center of Excellence lead\n

In the modern utility landscape, the shift from isolated legacy systems to interconnected digital platforms is no longer a future ambition—it's an operational necessity. System interoperability is the foundation for real-time decision-making, reliable service delivery, and customer satisfaction. As utilities modernize, the challenge isn’t just upgrading applications—it’s ensuring that core systems can communicate and scale together.\nCritical systems like NMS (Network Management System) for grid reliability, SGG (Smart Grid Gateway) for device communications, C2M (Customer to Meter) for customer and billing operations, and MDM (Meter Data Management) for data validation and aggregation each play vital roles, but when managed in silos, they limit both operational visibility and business agility.\nOracle SOA Suite empowers architects to break down these silos by establishing a flexible, service-oriented integration layer. This approach ensures seamless data flow, orchestrated business processes, and a scalable foundation that supports both today’s systems and tomorrow’s innovation across the utility enterprise.\nWhy SOA is the backbone of utility modernization\nAs utilities evolve to meet the demands of smart grids, distributed energy resources (DER), real-time customer engagement, and regulatory compliance, the ability to dynamically and reliably connect systems becomes the core of any successful architecture.\nOracle SOA Suite acts as the integration backbone, enabling architects to:\nDecouple tightly bound systems—reduce direct dependencies between platforms like NMS, SGG, C2M and MDM.\nStandardize data exchange—whether via SOAP, REST, JMS, or native Oracle adapters, ensuring smooth interoperability across legacy and cloud systems.\nAutomate end-to-end business processes—orchestrate complex workflows such as outage management, meter-data validation, billing cycles, and customer communications, all in real-time.\nEnforce security and governance—built-in policy enforcement, identity-aware connections, and monitoring provide architects with complete control over system interactions.\nWith SOA at the core, utility IT teams are no longer locked into rigid point-to-point integrations. Instead, they can design modular, reusable services that allow each system—whether NMS, SGG, C2M or MDM—to evolve independently while maintaining seamless connectivity.\nCommon integration patterns for utilities with Oracle SOA\nWhen designing modern utility architectures, the ability to handle diverse integration scenarios is critical. Oracle SOA Suite supports a wide range of patterns that address both real-time operational needs and long-running business processes.\nHere are some of the most common integration patterns utilities can implement with SOA:\n·      System-to-system orchestration
Use BPEL (Business Process Execution Language) to define and automate complex workflows involving systems like NMS, SGG, C2M and MDM. For example, when SGG receives a smart-meter event, SOA can route it to MDM for validation, forward it to NMS for outage detection, and notify C2M to update the customer record—all as part of a single orchestrated process.\n·      Event-driven integration
Leverage JMS, AQ or Oracle Event Delivery Network to publish and subscribe to system events. When real-time grid conditions change, such as a fault reported by NMS or a meter anomaly detected by SGG, SOA enables downstream systems to react automatically, minimizing manual intervention and improving response time.\n·      API Mediation and Transformation
Using Oracle Service Bus, architects can expose internal services as reusable APIs while transforming data between formats like XML, JSON, and proprietary utility schemas. This makes integrating external partners, cloud platforms, and new digital services into the existing utility landscape seamless and secure.\nSecure hybrid connectivity
\n
Oracle SOA Suite enables utilities to integrate cloud-native systems with on-premises platforms without exposing sensitive infrastructure. This is essential when bridging older CIS systems with cloud-based customer experience tools, or when connecting field devices to corporate data centers.\nBy leveraging these patterns, architects can ensure their utilities’ systems remain agile, reliable, and scalable—even as grid demands grow and new technologies emerge.\nReal-world use case\nWhen a smart meter sends an outage alert via SGG, Oracle SOA Suite enables:\nValidation of the event via MDM\nNotification to NMS for dispatch and restoration\nAutomatic updates to C2M for customer communication and service status updates.\nAll of this is orchestrated in real-time—no manual intervention, no system silos.\nFuture-ready architecture\nWhether you’re onboarding new DER systems, integrating Advanced Distribution Management Systems (ADMS), or expanding smart-grid capabilities, Oracle SOA Suite empowers utilities to evolve their technology stack without disrupting core operations.\nConclusion: Designing for agility, scaling for the future\nThe path to a future-ready utility isn’t paved with isolated applications or custom-built point-to-point connections—it’s designed with flexibility and interoperability at its core.\nOracle SOA Suite enables solution architects to move beyond reactive system maintenance and toward proactive service orchestration, real-time data flow, and business-process automation. Whether integrating core platforms like NMS, SGG, C2M or MDM—or preparing for new digital initiatives such as distributed energy resource management (DERMS) and advanced customer engagement—SOA offers the architecture to scale confidently.\nThe key takeaways:\n✔️ SOA transforms siloed systems into service-driven, reusable components.
✔️It enables real-time, event-driven and long-running process orchestration.
✔️System changes become less disruptive, supporting agile modernization.
✔️Oracle SOA Suite integrates cloud, on-prem, and hybrid environments securely.
✔️A SOA-based architecture prepares utilities for grid modernization and evolving
regulatory landscapes.\nIn short, from silos to synergy, Oracle SOA Suite is the foundation for designing resilient, future-proof utility ecosystems.\n\nNEED HELP UNDERSTANDING HOW THE ORACLE SOA SUITE CAN WORK AT YOUR ENTERPRISE? CLICK HERE TO CONNECT WITH US.\n\n


From Silos To Synergy: Oracle SOA Suite For Robust Utility-System Design

By Devika Kota, HEXstream data solutions engineer\nIn this blog post, I’ll explain how Databricks Delta Lake helps you turn raw or curated data into valuable insights for analytics.\nWhy use Delta Lake?\nDelta Lake takes traditional cloud data lakes to the next level by adding powerful features that solve common data challenges. Here’s a closer look at why Delta Lake stands out:\nACID transactions for data consistency—In many big-data systems, managing updates and writes during high-volume periods can lead to messy or inconsistent data. Delta Lake uses ACID (Atomicity, Consistency, Isolation, Durability) transactions, just like traditional databases, to make sure your data stays accurate and reliable—no matter how many users or processes are interacting with it at once.\nSchema enforcement to manage changing data—Data formats evolve over time. Sometimes new fields get added, or data types change. Delta Lake enforces schemas, which means it checks that incoming data matches the expected structure. This helps prevent errors and keeps your data pipeline stable, even as your source data changes.\nTime travel to track and recover data versions—Mistakes happen, whether it’s accidentally deleting data or running a faulty update. Delta Lake’s time-travel feature lets you easily view and restore previous versions of your data. This ability to “go back in time” helps with auditing, debugging and ensuring data accuracy over time.\nUnified support for streaming and batch data—Traditionally, streaming (real-time) data and batch (historical) data processing were handled separately, often requiring different tools. Delta Lake combines both into a single platform, enabling you to process live data and analyze historical data seamlessly. This unification simplifies your architecture and speeds up insights.\nMachine learning compatibility—High-quality, reliable data is the foundation for good machine-learning models. Delta Lake’s consistent and well-organized data makes it easier to build and train predictive models that can drive smarter business decisions and automate processes.\nBecause of these features, Delta Lake is especially useful for scenarios that require both quick, real-time data updates and deep, long-term analysis—like monitoring outages—where timely response and historical trend analysis are both critical.\nThe next section will walk you through five key steps to gain valuable insights from your data using Databricks Delta Lake.\nStep 1: Organize your source data with Unity Catalog\nYour source data might come from a database table or a file like CSV, Excel or text. The first step is to bring that data into Databricks using Unity Catalog. (Unity Catalog is a tool that helps you manage who can access your data and keeps everything organized and secure. It tracks data origins, usage, and permissions across teams and cloud environments.)\nHow to organize your data in Unity Catalog:\nCreate a catalog (think of it as a main folder for your data) \nDefine a schema (like a subfolder for a specific project)\nSet up a volume (where you'll store your actual files)\nUpload your CSV (or other) files to that volume\nStep 2: Convert data into a managed delta table\nNext, you can use a programming tool such as PySpark to read your files and convert them into what's called a "managed delta table." A delta table is a smarter data table in Databricks that’s faster and easier to manage. It tracks versions automatically, lets you update or delete data, and keeps your data consistent. It works well with both real-time and batch data.\nStep 3: Implement bronze, silver and gold layers\nThis step uses a popular method called the "medallion architecture," which organizes your data into stages for better management and clearer insights.\nBronze layer: Stores raw data as it is, including messy or duplicated info. This preserves everything for future use.\nSilver layer: Cleans and structures the raw data—removing duplicates, applying schemas, and joining sources. The data becomes more reliable.\nGold layer: Contains polished, aggregated data ready for dashboards, reports and machine learning. This is the version that decision-makers use for insights.\nThis approach keeps your data pipeline efficient and scalable.\nStep 4: Use the data for high-value outcomes\nWith your data refined and sitting pretty in the gold layer, you can now:\nBuild interactive dashboards using tools like Oracle Analytics Cloud, Power BI, Tableau or Databricks SQL\nMonitor live events with real-time visual reports that help catch issues quickly\nTrain machine-learning models on historical data to predict outages and optimize maintenance\nSupport smarter decisions with accurate, up-to-date insights on crew deployment, resource allocation, and planning\nBenefits of this approach\nSome major advantages you will gain by using Delta Lake:\nYour data will be consistent and reliable, even when dealing with massive amounts of information. \nYou can easily track changes for auditing purposes and keep a perfect history of your data. \nYou can handle streaming and batch data together, leading to much faster analysis. \nYou'll build scalable data pipelines that are ready for advanced analytics and machine learning. \nMost importantly, your teams will get quicker, clearer insights, empowering them to make much better decisions.\n\nNEED HELP WITH DATABRICK DELTA LAKE? CLICK HERE TO CONTACT US.\n \n


Tech Corner Tutorial: Using Databricks Delta Lake To Turn Raw Data Into Real Answers

By Swathi Ambati, HEXstream data solutions engineer\nThe APEX AI Assistant in Oracle APEX empowers developers to generate SQL queries using natural language, eliminating the need to memorize SQL syntax, table names, or complex relationships. Simply enter your question or request in plain language, and the assistant, powered by Oracle Cloud Infrastructure (OCI) Generative AI, produces accurate, syntactically correct SQL statements.\nIf the initial query isn’t exactly what you need, you can easily refine it by providing additional instructions, making the development process more intuitive and conversational.\nBut the capabilities of APEX AI Assistant extend well beyond SQL generation. Acting as an intelligent development partner, it can:\nExplain SQL or PL/SQL code in simple, understandable terms\nEnhance code readability and optimize performance\nIdentify and fix errors through contextual suggestions\nWhether you're crafting queries, debugging procedures, or learning SQL, APEX AI Assistant serves as an in-app mentor—offering real-time guidance, practical examples, and intelligent recommendations—all seamlessly integrated within the Oracle APEX App Builder.\nExample: I was reviewing some sample applications to understand how they work and customizing the queries to suit my requirements. The below video demonstrates one of the simple use cases to demonstrate NL2SQL using APEX AI Assistant. \n\n\nNEED MORE GUIDANCE ON USING ORACLE APEX? CLICK HERE TO CONNECT WITH US. \n
\n

Keep It Simple—Streamlining SQL And Code Development In Oracle APEX

By Bharath Moola, HEXstream data solutions engineer\nOracle Machine Learning for SQL, or OML4SQL, turns your Oracle Database into more than just a place to store data—it lets you do machine learning right where your data lives. Instead of exporting tables to external tools or notebooks, you can train models, test them, and make predictions all with ordinary SQL statements. \nThe benefits? This approach keeps your data safe inside the database, cuts down on copying files back and forth, and speeds up the process from raw numbers to real insights.\nHow it works \nBehind the scenes, OML4SQL integrates a handful of new, model-focused SQL statements—like CREATE MODEL and DROP MODEL—directly into Oracle’s SQL engine. When you issue a CREATE MODEL, the database’s parallel execution framework automatically splits your training data into chunks and processes them simultaneously across multiple CPU cores (or, in an RAC environment, across several nodes). \nThis true parallelism slashes training times compared to single-threaded approaches. And if you’ve enabled the Oracle In-Memory option, your tables can live in the In-Memory column store—compressed in columnar format and held entirely in RAM—so that I/O waits all but disappear and your model-building runs at maximum speed.\nRich, built-in algorithms\nOML4SQL gives you a wide selection of production-ready algorithms without any extra installations. For classification—like flagging high-risk grid segments—you can choose decision trees (ALGO_DECISION_TREE), random forests (ALGO_RANDOM_FOREST), logistic regression (ALGO_LOGISTIC_REG), support vector machines (ALGO_SVM), or Naïve Bayes (ALGO_NAIVE_BAYES). For numeric forecasting—such as estimating how long an outage will last—you can use linear regression (ALGO_LINEAR_REG) or support vector regression (ALGO_SVR).\nTo uncover common patterns in outage data—say, grouping incidents by equipment type, location, or cause—k-means clustering (ALGO_KMEANS) and hierarchical clustering help you segment your historical records. And for early-warning systems, one-class SVM (ALGO_SVM_ONE_CLASS) can monitor real-time sensor streams to detect anomalies that often precede unplanned outages.\nIn-database scoring\nOnce your model is ready, you don’t need a separate scoring system. You can add a prediction column to any SELECT query and get back results alongside your regular fields. If you’re running nightly reports, you can score millions of rows in one go. If you need near-real-time checks—say, flagging a questionable transaction—you can embed the scoring function directly into your application’s queries. \nEverything stays inside the database, which keeps things simple and secure.\nSecurity and governance\nSpeaking of security, OML4SQL fits right into Oracle’s existing controls. Models are stored in database tables, protected by the same user roles and encryption rules that guard your sensitive data. You get full audit trails that show who built each model and when they did so, and you can manage versions just like you manage table schemas; there’s no need to open another tool or platform to keep track of your machine-learning assets.\nWhy it matters \nIn everyday language, OML4SQL makes data science accessible to anyone who knows SQL. It removes the friction of moving data around, cuts down on extra servers or scripts, and brings predictive power into the heart of your operational database. \nIf you’re already comfortable writing queries, you can quickly add forecasting, classification or anomaly-detection to your toolbox—without learning a whole new environment. The result is faster insights, tighter security, and smoother collaboration between analysts, developers and decision-makers.\n\nINTERESTING IN CAPITALIZING ON MACHINE LEARNING AT YOUR ENTERPRISE? CLICK HERE TO CONNECT WITH US TO LEARN HOW.\n 

In-Database AI: Machine Learning With OML4SQL

By Siddartha Reddy Katukuri, HEXstream data integration engineer\nIn today’s AI-driven, API-first, real-time digital economy, utilities can’t afford to miss an event. Event-driven architecture is the nervous system of modern enterprises and it’s evolving fast.\nBusinesses are now demanding more than just data transport; we now need smart decision-making on every event. With the rise of generative AI, LLM-powered automation, and
real-time analytics, event data is no longer just "streamed," it’s interpreted, enriched and acted upon instantly.\nThis is a tremendous development. \nOracle Cloud Infrastructure (OCI) streaming offers a scalable, managed platform to fuel
these intelligent, high-velocity pipelines, and do so efficiently and cost-effectively.\nOCI streaming with Kafka: What’s new and why it matters\nOracle’s latest version currently supports Kafka 3.7 and a roadmap to adopt Kafka 4.0 soon means you get access to cutting-edge features:\nKafka Kraft mode—Eliminate the complexity of ZooKeeper. Kafka’s KRaft mode simplifies deployment, improves fault-tolerance, and is perfect for container-native environments like OCI Container Engine for Kubernetes.\nTiered storage—Optimize costs by moving cold data to OCI Object Storage, enabling longer retention without bloating your cluster.\nOracle is the smartest Kafka platform for AI\nOCI Infrastructure offers a fully managed interface, delivering all the tools you need under one platform without the operational complexity, which enables you to: \n·     Seamlessly use Kafka clients and ecosystem tools\n·     Dynamically scale with up to 24 brokers per cluster\n·     Secure with IAM, private endpoints, and audit trails\n·     Achieve OCI Data Flow (Apache Spark) for in-stream transformation\n·     Create autonomous databases and lakehouses in order to land, store and query stream data\n·     Use Oracle AI to apply GenAI and ML models to live event streams\nAI-ready streaming for personalization\nAI-powered customer experience, built on real-time Kafka infrastructure, is the future. With
OCI + Kafka: \n·     Users can clickstream events that are published in real-time\n·     OCI AI agents can enrich the data and score user intent\n·     Users can perform vector-search match behavior with past preferences\nWhy it's the right time\nOracle Cloud Infrastructure streaming cuts infra costs while scaling faster. It reduces integration complexity while enabling users to build real-time, AI-powered pipelines.\nIf you want to master events while enabling AI-first automation, it’s time to rethink how you stream. With Oracle Cloud Infrastructure, you gain the scalability of Kafka, the intelligence of Oracle AI, and the reliability of enterprise cloud all in one cost-effective platform.


Master Events Efficiently And Cost-Effectively With OCI

Our Swathi Ambati recently detailed how to accelerated APEX development with AI-driven functions in a \nTech Corner feature\n. Response to that post was enthusiastic, so we asked here to sit for a short Q&A exploring the wins that result from use of APEX. Take a look…\nHEXstream: APEX is ideal for rapid application development. What applications / solutions are ideal for this approach?\nSwathi:\n There are many: \nAutomation of internal workflows—APEX can replace ad-hoc Excel processes for everything from onboarding checklists to mobile-asset checks. Because of its speed, teams often implement these "shadow IT" solutions quickly to meet sudden demands.\nSharing information across departments—APEX applications can make curated data sets (e.g. historical outage data, environmental monitoring) accessible to non-technical users via simple web portals and reports, which bridges silos.\nData acquisition and management systems—With APEX we can streamline data-entry interfaces and effectively manage large data volumes.\nCustomer-facing portals—We can develop portals for customer support, service requests and feedback.\nHEXstream: All of these capabilities democratize application creation. How does that translate into wins for utilities? \nSwathi:\n APEX excels at rapidly building the kinds of applications that utilities frequently need. Its tight integration with Oracle Database means developers can quickly expose and manage existing data. Common high-impact use cases include:\nField-service coordination—The ability to deploy mobile-friendly applications for technicians. For example, field workers can use an APEX app to receive work orders, record inspection results, and capture digital signatures offline. Pre-built APEX components (lists, calendars, forms) speed this development, reducing paper processes and manual errors.\nCustomer self-service portals—We can quickly launch web portals where customers can view bills, report outages, request service, or track work orders. Because APEX apps are browser-based and responsive, utilities can prototype and roll out consumer-facing portals that integrate with billing and CRM systems with minimal code.\nWork order and preventive-maintenance systems: We can build applications to schedule recurring maintenance and better manage work orders. APEX’s low-code wizards enable utilities to define workflows (submit work requests, approved tasks, close work orders) in weeks instead of months.\nHEXstream: What are the most common uses of APEX that you experience? What is an intriguing, unusual use? \nSwathi:\n Common use cases I have experienced include: \nPrototyping and proof of concepts—Users can test ideas and demonstrate feasibility before committing to full-scale development.\nInternal tools and dashboards—Users can quickly build tools for project management, reporting and analytics tailored to specific business needs.\nPersonally, I have developed an APEX app as a knowledge portal where employees and developers can share custom solutions for the OOTB limitations and see the latest updates, for example features available, related blog articles published to APEX topics in social media or on the same portal as an APEX-related news bulletin.\nI recently came across an unusual use case in which APEX’s flexibility was employed at utilities; Oracle \npublished\n an architecture where APEX delivered interactive vegetation-management dashboards combining LiDAR images, AI predictions, and GIS layers to predict vegetation encroachment on power lines.\nHEXstream: What most excites you about the APEX development platform? \nSwathi:\n The combination of rapid delivery, enterprise scalability and low cost gives us confidence that new solutions will quickly pay for themselves. I am most excited about the APEX development platform for the following reasons:\nSpeed of innovation—APEX dramatically reduces development cycles. As a developer, APEX enables me to build robust applications with minimal coding, effectively leveraging SQL and PL/SQL skills.\nIntegration capabilities—Seamless integration with Oracle databases and cloud services enables unified data management and scalable solutions.\nCommunity and support—There is active community engagement and ongoing updates from Oracle, which ensure continuous learning and adoption of application-development best practices.\n\nCONNECT WITH US TO LEARN HOW APEX CAN BENEFIT YOUR ENTERPRISE. CLICK HERE TO CONTACT US.\n 

Speed, Integration And Community: A Q&A On The Best APEX Capabilities

By Krishna Sai Geetla, HEXstream data analytics engineer\nOverview: \nThe use case showcases support and the technical flow for external datasets to be combined with out-of-the-box data and build reports/visuals in OAC. All customizations are performed in Fusion Data Intelligence console’s semantic-model extensions. For the external data, a table is created in the target ADW database and is read in semantic model extensions of FDI.\n

GOAL\n: To verify if a new dimension with third-party data can be added to out-of-the-box subject areas' fact and verify data in visuals/reports when the data from custom dimension is selected.

\nKey points to consider:\n
→ Third-party data source for this use case is considered as dataset coming through an Excel sheet.

→ There must be a custom schema created in target ADW database with its own user-login details. 

→ The new table must have matching primary key data to join with the out-of-the-box Fact.

→ Both the primary key names for existing Fact and new dimension must not be same.

\nFlow of steps: \n
→ Get the data set through an Excel sheet and import to target database.

→ Give necessary grants to OAX_USER and OAX$OAC users on the custom table.

→ In FDI platform, create a sandbox by modifying the existing logical star of the required subject area through the semantic-model extension feature.

→ Modify out-of-the-box subject area and apply changes.

→ For publishing the changes, this current flow used publishing of sandbox to directly verify in OAC. If user wants to make changes permanently, we can merge to main sandbox and publish main with new changes.

→ After publishing, check the newly added changes are reflected in the existing folders.

→ Final step is to verify the new dimensions data is available in OAC reports. Also, to verify if the new dimension columns yield data when combined with existing fact and dimensions.

\nDetailed steps:\n·      Connect to the target database with custom user credentials.\n·      To import the new data from Excel sheet, right click on “Tables” and then select “Import Data”.\n\n\n\n\n·      Select the file with data.\n\n\n·      Select format as Excel 95-2003(xls) if it is not automatically and click “Next”.\n\n\n·      In next wizard, select import method as Insert.\n·      Give the table name and click “Next”.\n\n\n·      Choose the columns from excel sheet that needs to be attributes in new table in DB and click “Next”.\n\n\n·      Change datatypes and size as per requirement or according to the columns in existing fact.\n·      Both the primary key names for existing fact and new dimension must not be same.\n·      The data type of primary keys of the two joining entities (custom & existing) must be the same.\n\n·      Click on "finish" in the import summary wizard.\n\n\n·      Click on “ok” on the import message.\n\n\n·      For the “OAX_USER” to read/edit the data in new custom dimension in custom schema, there must be grants on the new table. \n·      In OAC “OAX$OAC” must have read access to the custom table.\n·       Use the following grants:
GRANT SELECT ON CUSTOM_EWDIDEV2.CUSTOM_HOUSEHOLD_PAYMENT_BURDEN TO OAX$OAC; GRANT ALL ON CUSTOM_EWDIDEV2.CUSTOM_HOUSEHOLD_PAYMENT_BURDEN
TO OAX_USER;\n\n\n·      Check the data in new table after importing.\n\n\n·      In Admin console, go to Semantic Model Extensions.\n\n\n\n·      In user extensions select create sandbox and enter a name.\n\n\n\n\n\n·      Select “Manage logical star” in “perform action” dropdown.\n\n\n·      Select the edit logical star->out of the box->Subject area name-> Fact name and click on
“Next”.\n\n\n\n·      In the logical star structure click on “manage dimension” drop down and select “add dimension”.\n\n\n·      Select the custom schema->Object (table name)->Dimension name is auto generated(use different if the name is already used).\n·      Check on the column attributes that needs to be in the new dimension and click “Next”.\n\n\n\n·      Uncheck hierarchy->give hierarchy name(mandatory)->select the detail primary key and display attribute.\n\n\n\n\n\n\n\n\n\n·      Click on “Finish”.\n\n\n·      Join the fact and custom dimension by dragging the connector from fact to dimension.\n\n\n·      Select the appropriate join and fact-dimension joining keys.\n\n\n\n\n\n\n·      Select the content level (mandatory) and click “Done”.\n\n\n\n·      Validate and go back.\n\n\n·      After going back click on perform action to modify subject area.\n\n\n\n·      Select the required subject area and click “Next”.\n\n\n·      Verify subject area folders and click “Next”.\n\n\n\n·      In the second Wizard click manage elements dropdown and select Manage new customizations.\n\n\n·      Click on the first check box to select all of them (we can select individual labels as per requirement). \n·      Click Add.\n\n\n\n\n\n\n·      Verify all new custom changes are in folder and click Finish.\n\n\n\n\n·      Go back to user extensions to apply sandbox changes.\n\n\n\n\n\n\n\n\n\n·      Publish the model after confirming sandbox changes.\n\n\n\n\n\n\n·      Go to OAC after publishing of model.\n\n\n\n·      Select the subject area from Data page form dropdown.\n\n\n\n·      Verify the new dimension folder shows up.\n\n\n\n·      Select the columns from dimension to verify data.\n\n\n·      Verify the connection of new dimension with existing fact/dimensions by adding columns from out of the box fact and dimensions.\n\n\n
\nCLICK HERE TO CONNECT WITH US ABOUT HOW OUDI CAN OPTIMIZE YOUR PROCESSES. \n

Tech Corner Tutorial: Modify Existing OUDI Subject Areas By Adding New Dimensions With Third-Party Data As The Source

By Ashwini Nagendra Prasad, HEXstream Integrations Center of Excellence lead\nAs utilities navigate the massive shifts in energy—from centralized to decentralized, from fossils to renewable, from passive consumption to active participation—one thing has become painfully clear: our energy systems aren’t talking to each other fast enough.\nWe’ve got legacy infrastructure trying to sync with modern DERs. We’ve got microgrids, EVs, storage systems, and smart meters popping up like mushrooms after rain—but stitching all these pieces together into a unified, real-time grid?\nThat’s the hard part. And it begs the question: Is Oracle’s Live Energy Connect (LEC) the missing link?\nWhat is Live Energy Connect (LEC)?\nOracle LEC is a high-performance, vendor-agnostic middleware platform designed to simplify and streamline grid data integration in real time.\nIn non-tech speak, it’s like a universal translator and traffic controller for utility systems—SCADA, EMS, DERMS, substations, cloud apps, IoT, you name it. LEC connects the dots between:\nLegacy OT systems and modern IT platforms\nField devices and control centers\nDistributed assets and centralized operations\nInternal systems and external third parties (market operators, aggregators, etc.)\nIn essence: It brings intelligence to the edge and interoperability to the core.\nWhy t's potentially the missing link\nUtilities aren’t lacking in data—they’re drowning in it. What they lack is real-time interoperability and actionable insights. LEC helps solve that in a few big ways:\nReal-time data translation: LEC translates and routes data between hundreds of proprietary protocols and systems, eliminating silos and vendor lock-in.\nDER integration without chaos: It enables fast, scalable integration of DERs—from rooftop solar to EV fleets—into existing operations without reinventing the entire grid.\nEdge intelligence: With logic at the edge, LEC allows for local automation and faster decision-making (ideal for substation automation and microgrids).\nGrid-modernization enabler: It’s not just a patch—LEC becomes the connective tissue for broader digital transformation strategies.\nThe challenges utilities face without it\nWithout a platform like LEC, utilities often struggle with:\nOutdated protocols that can’t scale\nComplex, brittle custom integrations\nLimited visibility into DERs and field assets\nDelayed control signals and data overload\nIn a world moving toward transactive energy and real-time balancing, that just doesn’t cut it.\nWhy LEC matters right now\nThe pressure is on. Grids are expected to be smarter, more flexible, cleaner and more secure. But no amount of hardware or policy can deliver all that unless your systems can communicate in real time across vendors, platforms, and layers.\nThat’s what LEC enables. It’s not just a data integration platform—it’s a strategic capability.\nFinal thought: Modern grid, modern glue\nAs the energy sector gets more complex, interoperability is no longer optional—it’s mission-critical. Oracle Live Energy Connect might not be the flashiest piece of tech in the utility stack, but it could be the most important one you don’t see. Because when everything’s connected, everything works better.\nSo yes...maybe it \nis\n the missing link.\n\nCLICK HERE TO CONNECT WITH US ABOUT LEC AT YOUR UTILITY.\n \n


Utilities Unplugged: Is LEC The Missing Link?

By Praveen Reddy Atturi, HEXstream senior data analytics engineer\nLoading Data and Metadata to Hyperion Essbase Application (Oracle SQL to Essbase Application)\nEssbase is a multidimensional database that enables business users to analyze data from multiple perspectives and at a various level of consolidation. It maintains parent-child hierarchies and stores data in a multidimensional array format.\nFor Example, Time based hierarchies may include levels such as \nMinute → hour → Day → Week → Month → Quarter → Half Year → Year.\nTo begin working with Oracle Data Integrator (ODI), the first step is to launch the ODI studio application and connect to the repository. Follow these steps get started. Go to start à ODI and login to work repository.\n1. Topology setup in ODI\nAfter successfully logging into work repository go to the Topology Navigator select Physical Architecture Tab, Select Technology and expand it. GO to the Hyperion Essbase Right Click and select New Data Server. In the Definition tab enter the details of the server and provide the username and password.\n\nClick on save it will ask you to create physical schema press OK.\nIn the Technology tab go to Hyperion Essbase expand it and select the newly created data server Right Click and Select New Physical Schema. In the definition tab enter the Essbase Application name and database name to which you want to connect.\n\nClick on save It will ask you to specify the context press OK.\nGo to the Logical Architecture tab and expand technologies and select Hyperion Essbase. Right Click and select New Logical Schema.\nGive Proper name to logical schema and set the Context to Hyperion Essbase physical schema.\n\nClick on Save.\n2. Import knowledge modules\nExpand Project go to knowledge module right click and import the below knowledge modules.\nRKM Hyperion Essbase, LKM Hyperion Essbase Metadata to SQL, LKM Hyperion Essbase Data to SQL, IKM SQL to Essbase Metadata to SQL and IKM SQL to Essbase Data to SQL.\n\n3. Reverse engineer\nNow go to the model tab and create a new model folder. Now I will go to the newly created Model folder and create a new model. In the definition tab enter the name of the model and select the technology, select the logical schema, and save it.\n\nThen go to the Reverse engineer tab Select customize check box and select the Context, Logical Agent, and knowledge module. Click save.\nNow newly created Model will be available for Reverse Engineering. Go to Model tab select Model and Right click and select Reverse engineer. Before Reverse Engineer the models make sure your agent services are running if they do not start it.\n\nNow we can reverse engineering it. Click on the reverse engineer, select the context, Agent and log level, the click on OK.\nNow Go to the Operator Tab to check whether the execution run successfully or not. If all the checks are green that means your model reverse engineered successfully. Now expand the Hyperion Essbase model, and you can check all the Dimensions are visible there from your Essbase Application.\n4. Creating a project folder and developing ODI mappings\nNow, Let’s create new project and within it, a create new mapping. Go to mappings section, right click and select New Mapping. Enter the suitable name for mapping.\nGo to the mapping tab and drag the table from oracle data Model to source area and drag the Essbase model and drop it to target place.\nNext Select the target object and select LKM & IKM.\n·       In the logical area, the LKM was set to select LKM SQL Multi-Connect.Global.\n·       IKM was set to IKM SQL to Hyperion Essbase (Metadata). — Loading Metadata to Hyperion Essbase Application \n·       IKM was set to IKM SQL to Hyperion Essbase (Data). — Loading data to Hyperion Essbase Application.\nRule File: Need to specify the name of the Rule File to be used for the Metadata load. This Rule File was created at the Essbase Application level.\nRule File separator: Rule separator specified in the rule file (Optional).\n·       Valid Values- Comma, Space, Tab and Custom characters (^, #, @).\nLOG_FILE_NAME: If you want to see the logs showing how many records were successfully inserted or rejected, we need to create a specific path and specify that path in the log file name.\nERROR_LOG_FILE_NAME: If you want to view the error records and understand the root cause of the errors, you need to create an error.log file for that specific mapping in a designated path and provide that path in the ERROR_LOG_FILENAME parameter.\nØ  IKM SQL to Hyperion Essbase (Metadata Load) for Essbase data load.\n\nØ  IKM SQL to Hyperion Essbase (DATA) for Essbase data load.\n\nThen, click on the save button.\n5. Execution\nExecute the newly created mapping. Then go to the operator navigator and check if the
execution is successful or not.\n\nThen go to your target application and check the data load.\nLoading data and metadata to Hyperion Essbase Application (flat file to Essbase Application)\n\nI am going to be using the data server and physical schema I have set up previously connecting to the Database.\nThe first step to Create a model to hold the reversed Essbase database. If you already have an existing model, you can reuse it and perform the reverse engineering again.\nNow, we can proceed with the Reverse engineering process. Click on Reverse Engineer. Select the appropriate context, Agent and Log level, then click on OK. Navigate to the
Operator tab to verify whether the execution was completed successfully or not. If all the indicators are green, it means the model has been reverse engineered in all the dimensions successfully. You can then expand the Hyperion Essbase model to view all the dimensions retrieved from your Essbase Application.\n\nNow, a CSV file needs to be created, and reverse engineered. The file should be saved in the directory specified by the file technology configuration, in this case the system directory (C:\Oracle\Middleware\Oracle_ODI\Flatfile etc....)\nA Datastore was then created using the Flat file model. To do this, navigate to the model
section and Create a New Model. Select Technology and logical schema. Then save the configuration.\n\nNext, right click on the newly created model and select New datastore and enter desired the
name and alias, then click on resource name search field to locate and select the previously created CSV file, from here you get the created CSV file. \n\nThen, navigate to the Files tab to specify the record separator either MS-Dos or Unix. Next, go to the 'File Format' section and choose the appropriate format (\nDelimited or fixed).\n Set the header row accordingly, and define the field separator \n(tab, space, comma or any other character as required). \nThen specify the text delimiter by setting it to a double quotation (").\n\nThe Datastore was then reverse engineered to generate the columns definitions based
on the structure of the Flat file.\n\nSo, that’s the source and target models creation done, time to create mappings.\nNow, let’s create new project and within it, a create new mapping. Go to mappings section, right click and select New Mapping. Enter the suitable name for mapping. Go to the mapping tab and drag the Source file from Flat file Model to source area and drag the Essbase model and drop it to target place. Then map the columns between source and target. In the logical select the Target integration as Control append then, go to the physical tab and select the Target object and:\n·       In the logical area, the LKM was set to LKM file to SQL as we are loading to flat file (Source to Staging Area load).\n·       IKM was set to IKM SQL to Hyperion Essbase (Metadata). - Loading Metadata to
Hyperion Essbase Application \n·       IKM was set to IKM SQL to Hyperion Essbase (Data). - Loading data to Hyperion Essbase Application \nRule File: Need to specify the name of the Rule File to be used for the Metadata load. This Rule File was created at the Essbase Application level.\nRule File separator: Rule separator specified in the rule file (Optional).\n·       Valid Values- Comma, Space, Tab and Custom characters (^, #, @).\nLOG_FILE_NAME: If you want to see the logs showing how many records were successfully inserted or rejected, we need to create a specific path and specify that path in the log file name.\nERROR_LOG_FILE_NAME: If you want to view the error records and understand the root cause of the errors, you need to create an error.log file for that specific mapping in a designated path and provide that path in the ERROR_LOG_FILENAME parameter.\nØ  IKM SQL to Hyperion Essbase (Metadata Load) for Essbase data load.\n\nØ  IKM SQL to Hyperion Essbase (Data Load) for Essbase data load.\n\nClick on the save button, Execute the newly created mapping. Then go to the operator navigator and check if the execution is successful or not. Finally, go to your target application and check the data load.\n\nNEED HELP? CLICK HERE TO CONTACT US ABOUT STRATEGICALLY LOADING YOUR DATA. \n

Hyperion Essbase Integration With Oracle Data Integrator (ODI)

By Deepika Geetla, HEXstream data engineer\nIn the world of big data, efficient ETL (extract, transform, load) workflows are the backbone of data-engineering operations. As businesses generate massive amounts of data, optimizing these workflows for scalability, reliability and speed is crucial. \nA powerful way to achieve these goals is by leveraging Databricks with Delta Lake, which together provide a robust solution for modern ETL pipelines.\nIn this article, we will explore how Databricks and Delta Lake work together to enhance ETL processes, ensuring high performance, data consistency and scalability.\nWhat is Databricks\nDatabricks is a unified data-analytics platform built on Apache Spark, designed for efficiently handling large-scale data workloads. It supports both batch and real-time data pipelines, making it a preferred choice for data engineering, machine learning and analytics.\nWhat is Delta Lake\nDelta Lake is an open-source storage layer that enhances data lakes with ACID transactions, schema enforcement and versioning. It is built on Apache Parquet, making it highly optimized for analytical workloads. Delta Lake ensures that ETL workflows maintain data integrity, reliability and performance at scale.\n\nWhy use Databricks and Delta Lake together?\nWhile Databricks provides high-speed data processing, Delta Lake improves data quality, consistency and management. By integrating the two, organizations can:\nEnsure data reliability with ACID transactions\nOptimize performance through indexing, caching and optimized storage\nEnable incremental processing instead of full reprocessing\nSupport time travel for rollback and historical analysis\nManage both batch and streaming data seamlessly\n\n\nAdvantages of using Databricks with Delta Lake\n1. Ensuring data quality and consistency\nACID transactions: Ensures atomicity, consistency, isolation and durability, preventing data corruption\nSchema enforcement: Automatically prevents unexpected changes in data structure\nData validation: Guarantees that only clean, structured data enters the pipeline\n2. Boosting performance and reducing costs\nTraditional data lakes often suffer from fragmentation, leading to slow queries and high costs. Delta Lake solves this with:\nCompaction of small files: Organizes data into efficient storage formats, reducing latency\nIndexing for faster queries: Uses techniques like ZORDER clustering to speed up searches\nCaching and data skipping: Reduces the need for scanning entire datasets\n3. Enabling incremental data processing\nInstead of reloading entire datasets, Delta Lake supports:\nMerge operations: Updates only changed records, improving efficiency\nChange data capture (CDC): Detects and processes only new or modified records\n4. Supporting Time Travel and data recovery\nOne of Delta Lake’s standout features is Time Travel, which enables users to access previous data versions.\nRollback mistakes: Instantly revert to an earlier version if errors occur\nData auditing: Keep track of historical changes for compliance and governance\nTactics for optimizing ETL workflows with Databricks and Delta Lake\nData ingestion: Load data from multiple sources (cloud storage, databases, APIs) into Delta Lake tables. Use auto-loader in Databricks for efficient streaming data ingestion.\nData transformation: Perform cleansing, deduplication, and aggregations using Spark SQL and Delta Lake. Optimize transformations with Partitioning and ZORDER indexing.\nData loading: Store transformed data into Delta format for efficient querying. Use Databricks SQL for reporting and analytics.\nScheduling and orchestration: Automate ETL workflows using Databricks Workflows or Apache Airflow.\nMonitoring and error handling: Use Delta Lake transaction logs to track changes and debug failures. Implement alerting mechanisms for failed jobs.\nBest practices for optimizing Databricks and Delta Lake ETL workflows\nPartition data efficiently: Helps in parallel processing and improves performance\nUse delta caching: Reduces query execution time by storing frequently accessed data in memory\nLeverage auto-scaling: Dynamically adjust cluster size based on workload to optimize costs\nOptimize storage format: Store data in Parquet or Delta format for better compression and performance\nEnable data lineage tracking: Maintain audit logs for better governance and compliance\nConclusion\nBy combining Databricks for data transformations with Delta Lake’s storage and reliability features, organizations can build highly efficient and scalable ETL workflows. This integration enables automated data processing, ensures data quality and reduces costs.\nWhether dealing with batch or streaming data, using Databricks and Delta Lake together is a significant evolution for modern data engineering.\n\nCLICK HERE TO CONNECT WITH US TO DISCUSS OPTIMIZING YOUR ETL WORKFLOWS.\n 

Optimizing ETL Workflows Using Databricks And Delta Lake

By Satish Saini, HEXstream utility industry specialist\n\nOur dependency on electrical power is growing rapidly. Demand across the globe is rising.  Electricity plays a significant role in the development of industry, public infrastructure and the overall GDP of country, but most utilities delivering that power are facing lots of challenges with sustainability, reliability and controlling costs. \nThe World Bank reports that: \nElectrical utilities’ current operating model is facing unprecedented challenges.\nSome 60% of utilities in developing countries are not financially sustainable and not able to cover their operating and debt-service costs, which jeapordizes their energy-transition and access goals.\nThese financial and operational hurdles deter investors, preventing many utilities from critical investments in grid modernization and necessary upgrades to aging infrastructure. This also leads to an increased reliance on government subsidies and write-offs, eventually straining national budgets and diverting finances from other critical investments.\nMany utilities face acute challenges like high operating costs, greater transmission and distribution losses, lower reliability, underperformance of assets, inefficient meter-to-cash process and many more.\nIneficiencies and power losses can eliminate up to 4% of the GDP in some countries.\nThe good news \nWe have technical solution to address these losses. We have \nposted earlier\n about utilities transmission and distribution losses (both technical and non-technical), including methodologies for accurately detecting/quantifying/reporting these losses (especially non-technical).\n\n\nNon-technical losses or commercial losses that occur due to factors beyond technical issues with the grid, can be classified as:\n—Electricity theft, unauthorized consumption of electricity\n—Fraud, intentional misrepresentation or manipulation of energy usage and information\n—Tampering with meters to inaccurately record consumption\n—Inefficient “meter-to-cash” processes: loopholes, inefficiencies and problems in the end-to-end process of energy accounting and revenue collection from sale of energy to consumers, comprising the following activities cycle:\nMeter-reading process and relevant issues and loopholes\nBilling and invoicing process errors and issues, including unbilled revenue\nPayments and bill-collection process issues, including unpaid and unrealized revenue\nInvestigation and enforcement process and KPIs for unbilled and unpaid revenue\nRevenue reporting and financial statements, anomalies and exceptions\nProactive actions for the next billing cycles\nIn future installments in this series, we will dive deeper into non-technical losses, especially under the \n“\nmeter-to-cash (M2C)” process inefficiencies and loopholes topic. The next segment will cover specific issues related to each activity under the full meter-to-cash cycle, including meter reading, billing and invoicing, payments and receipts with reflection on utility revenue and financial statements. And later segments will cover the relevant KPIs and steps to reduce M2C losses under each activity in a billing cycle by taking proactive steps with  auditing, investigation and enforcement.\n\nClick here to contact us\n to learn how an efficient “meter-to-cash” process can help your utility reduce unbilled and unpaid revenue to enhance your financial sustainability. \n

Fixing Sustainability And Operating Losses Throughout The Meter-To-Cash Process

\n\nBy Srivyshnavi Akarapu, HEXstream data analytics engineer
\n
In this blog, we will discuss how to use Fusion Data Intelligence (FDI) to build a new subject area in Oracle Utilities Data Intelligence (OUDI). We will demonstrate how to create dimension and fact tables using data augmentation, enabling improved analytics and more thoughtful analysis.\nData Augmentation: FDI data augmentation uses the FDI data pipeline to enrich reports with new dimensions and fact tables, thereby improving business analytics. With the use of this capability, businesses may include more data sources, leading to deeper insights and better decision-making inside their target systems.\nCreate fact augmentation\n·    Sign into oracle fusion data Intelligence, In Oracle Fusion Data Intelligence console:\n·     Go to Data Configuration under Application Administration.\n·     On the Data Configuration option, click Data Augmentation.\n\n·     Select Augmentation type as create fact.\n·     Select source dataset type as supplemental data.\n·     In source table type specify the source table.\n\n·     In the next step, select the attributes from the source table that you want in the new fact table, then click Next.\n·    Select at least one attribute as a measure to proceed.\n·     In the next wizard, provide the below details and click Finish to save. \n·     Name the augmentation.\n·     Enter a description of the job.\n·     Give suffix for the target table name, augmentation process automatically creates the target table.\n·     Specify whether to save the data augmentation pipeline job without running it, or schedule the execution time and date, or run it right away.\n\n

\n     Create dimension augmentation\n·     Select Augmentation type as create dimension.\n·     Select source dataset type as supplemental data.\n·     In the source table type specify the source table.
\n\n
·     In the next step, select the attributes from the source table that you want in the new dimension table, then click Next.\n·     In the next wizard, provide the below details and click Finish to save. \n·     Name the augmentation.\n·     Enter a description of the job.\n·     Give suffix for the target table name, augmentation process automatically creates the target table.\n·     Specify whether to save the data augmentation pipeline job without running it, or schedule the execution time and date, or run it right away.\n\nSemantic model extensions
\n
The Semantic Model in Oracle OUDI and Fusion Data Intelligence (FDI) provides the flexibility to create new models or modify existing ones by adding additional attributes from different data sources. It supports Semantic Model Extensions, enabling businesses to introduce new attributes, measures, and subject areas without altering the core data model.

Organizations can create a centralized, consistent data structure using the Semantic Model, which will support data-driven decision-making and provide users with self-service analytics.\nCreate a logical star \n·     Under the user extensions tab in the semantic model, create a sandbox. \n·     In a sandbox, click perform action, create logical star.\n\n
·     Select Add fact option and choose the appropriate schema and object to the logical star.\n·     Name the fact table and select the required columns and keys.\n·     Apply suitable aggregations on  columns.\n\n


\n  Add dimensions\n·     Click on Manage dimensions, select add dimension option.\n·     Give the appropriate schema and object and add the dimension.
\n\n

\n\n   Define hierarchies\n·     Name the hierarchy and add levels to your hierarchies by right click on parent folder, add Level or add n Levels.\n·     Drag required data elements from "Available Data Elements" to the Level structure.\n·   Click on level folder structure and add relevant primary keys to each level to proceed to Finish.  
\n\n·     Join the fact to the dimension by giving suitable join condition.\n\n
\nCreate a new subject area\n·     In perform action select Manage subject area, create a subject area.\n\n\n·      In manage elements option, select manage new customization ,add fact and dimension. \n\n

\n\n·       Verify the final structure and click on finish.\n\n

\n\n·     To publish these customizations in the system we need to follow the steps below on the sandbox action menu.\n·     Apply changes\n\n
·   Publish the model using main user extension.


\nCLICK HERE TO CONNECT WITH US ABOUT OUDI.\n\n\n
\n

Tech Corner Tutorial: Creating A New Subject Area In OUDI Using Data Augmentation And Semantic Model Extension

By Pavani Dumpala, HEXstream data analyst \nIn Power BI, row-level security (RLS) plays a pivotal role in restricting data access at the user level, ensuring that individuals only see the data relevant to their role or permissions when interacting with the reports. One of the most powerful and flexible implementations of RLS is dynamic row-level security, particularly when managing complex organizational structures.\nFor instance, a manager can view data for their team, but they won’t be able to access information from other departments. This ensures that each user only sees the data relevant to them, keeping sensitive information secure and maintaining data privacy across the organization.\nIn this blog, we will explore dynamic row-level security within the context of an organizational hierarchy, using Power BI’s capabilities to tailor data visibility for users at different levels. \nOrganizational hierarchy\nTo manage and analyze hierarchical data in Power BI, where rows are linked by a parent-child relationship, specialized parent-child DAX (Data Analysis Expression) functions play a crucial role. Some commonly used functions for managing hierarchical data include:\nPATH:\n Returns a delimited text which contains all the parents to the current row.\nIn our example, on a table that contains employees, the managers of employees, and the managers of the managers, you can return the path that connects an employee to his/her manager. \nDAX Syntax : PATH(Child ID, Parent ID) \nPATH CONTAINS: \nChecks whether a specified identifier exists within the hierarchy path.\nDAX Syntax : PATH CONTAINS(PATH(Child ID, Parent ID),Parent ID))\nThis checks if a specific parent is part of the hierarchy chain for the child.\nPATH ITEM:\n Helps in extracting an item at a specific level within the hierarchy.\nDAX Syntax : PATHITEM(PATH(Child ID, Parent ID), 3)\nThis returns the item at the third level of the hierarchy for a given child from the top.\nPATH LENGTH: \nReturns total number of levels in the hierarchy path generated by the PATH function.\nDAX Syntax: PATHLENGTH(PATH(Child ID, Parent ID))\nExample scenario\nIn this scenario, we explore how to implement dynamic row-level security (RLS) in Power BI for an organization that tracks both employee hierarchy and sales performance. The goal is to ensure that each employee (particularly managers) can view the sales data of their team while restricting access to only relevant data based on their position in the hierarchy. Employees should not be able to view their managers data.\nData structure\nEmployee table contains the following:\n\nSales table contains the following:\n\nThe employee and sales tables are linked via the employee ID column.\nRequirement\nEmployees can only view their own sales data.\nManagers can view their team’s sales data, including any employee reporting to them at
any level.\nNo employee should be able to view their manager’s sales data.\nImplementation\n1. Employee table setup: Add a calculated column to the employee table to generate the hierarchy path for each employee, which shows the chain of managers above them. You can use the PATH function to create a UserPath: \n\nThis creates a path that shows all the employee IDs of the managers for each employee.\n2. Defining RLS in Power BI : The RLS is implemented through DAX expressions in the “Manage Roles” section of the Power BI. When an employee signs in, the model dynamically adapts to the logged-in user’s email address, ensuring data is filtered according to their role. The logic also ensures:\nAn employee can see only their own sales data.\nA manager can see their team’s data but not their manager’s.\nTo do this go to Modeling>Manage Roles>Create new role. Here a new role created named Dynamic RLS and under Employee Data created the DAX.\n\nUSER PRINCIPLE NAME: Returns the logged-in user’s email address.\nFilter: Filters the employee table to identify the row corresponding to the logged-in user based on their email ID.\nMAXX: Retrieves the employee ID of the logged-in user.\nPATH CONTAINS: Checks if the logged-in user’s employee ID is contained within the UserPath of other employees. This ensures the user can only see data for employees reporting to them.\nResult\nNow when a manager logs-in, Power BI recognizes the hierarchy defined by the UserPath and allows the manager to view sales data for all the employees reporting to them, including direct and indirect reports.\nHere under the “View as” section I have given an email ID of a manager, and the results are as below giving the employees under that manager.\n\n\n\nThe roles are initially defined in Power BI Desktop, and once the report is published, they are enforced in Power BI Service to control data access dynamically.\nTo do that, in Power BI Service navigate to Dataset’s security settings where you can assign users or groups to the RLS roles. By doing this, we are applying data access restrictions at the dataset level. \nFinal insights\nThis setup ensures that data access is dynamically controlled based on the logged-in user’s email. This enables organizations to securely manage access to sensitive data based on an employee’s role and reporting structure. \n\nCLICK HERE TO CONNECT WITH US ABOUT OPTIMIZING YOUR POWER BI STRATEGIES. \n

Achieving Dynamic Row-Level Security (RLS) With Organizational Hierarchy In Power BI

By Satish Saini, HEXstream utility industry specialist \nWater-supply utilities play a vital role in providing potable, clean, safe and reliable drinking water to communities, while also protecting the environment. The whole process ensures public health and environmental sustainability through raw water sourcing, filtering, treating and distributing potable water through a vast and widely scattered water-supply infrastructure. \nIf you work in this space, you understand the complexity involved. And as a water consumer, you appreciate service so reliable that you rarely have to think about it. \n\nIssues and impacts:\n The whole US water-supply system is facing issues related to high levels of losses through leakages, system breakdowns, and unauthorized usage. Consider also the aging water-supply infrastructure and the slow adoption of measurement/monitoring/automation technologies in water-flow analytics and loss detection. These factors undermine reliability and system performance, and increase O&M costs for utilities. There there is the misuse of natural resources and poor economic performance, which can lead to utilities missing sustainability and profitability targets.\nInfrastructure:\n The key components of this complex system include raw water sources including reservoirs, lakes, rivers, dams, water wells, etc. Water-intake pumping systems take this raw water to water filtering and treatment plants, along with supplies further through pumping systems, reservoirs and watermains piping to communities. There it (again) goes through field/community pumping stations, reservoirs and is then delivered to final end consumers through distribution piping systems and customer-service lines.\nWater losses:\n As reported years ago in a World Bank study, every year more than 32 billion cubic meters of treated water physically leak from urban water-supply systems around the world, while 16 billion cubic meters are delivered to customers for zero revenue. And a similar UNICEF report noted that many Middle Eastern and North African countries are at extreme risk of water scarcity in the near future. And the American Water Works Association (AWWA) reports that most of USA’s drinking water infrastructure/pipes will need to be repaired or replaced by 2040.                           \nThese high levels of losses and our aging water-supply infrastructure are triggering the need for not only upgrading systems, but also embracing technologies like advanced metering, monitoring and sensing devices, digitization and digitalization with AI and data-driven analytics and insights on assets performance and system losses.\nOur needs for water are not going to change in the near future. Our methods for delivering that water must. \n*See the first installment in this water series \nhere\n. In our next water fearture, we'll focus on the types of losses, including physical losses, apparent losses and non-revenue water (NRW). We'll highlight emerging technologies to detect and reduce losses for improved overall system efficiency and utility sustainability and profitability.\n\nCLICK HERE TO CONNECT WITH US ABOUT HOW DIGITALIZATION CAN HELP YOUR UTILITY MANAGE YOUR WATER-SUPPLY INFRASTRUCTURE AND OPTIMIZE YOUR PROCESSES.\n

Utility Water Supply Systems—Losses, Analytics & Insights

By Faisal Babu, HEXstream senior data analytics engineer \nToday’s fast-paced world, utilities—whether they’re in the water, gas or electricity sector—must harness the power of data to make informed, strategic decisions. Oracle, with its comprehensive suite of solutions, has become a pivotal player in the utility sector, enabling organizations to optimize operations and drive efficiency through powerful analytics and reporting capabilities. \nOne of the most impactful elements within Oracle Utility Analytics is data visualization, a critical tool that transforms complex datasets into intuitive, interactive and actionable insights.\nIn this blog, we will explore the significance of data visualization in Oracle Utility Analytics Reporting, highlight its benefits, and explore how utilities can leverage it to enhance operational efficiency and customer satisfaction. We'll look at specific applications of Oracle Utility Analytics’ visualization reporting and show why it’s a game changer for utilities.\nA game-changer for utilities\nThe utility industry is undergoing a transformation. From smart meters to advanced analytics, technology is playing an increasingly important role in optimizing utility operations. A key feature of Oracle Utility Analytics is visualization reporting, which turns raw data into actionable insights through visual representation.\n\n\nOracle Utilities Analytics visualization provides prebuilt data areas and visualizations for utilities' data. The Oracle Analytics platform included within your Oracle Utilities SaaS app enables deeper data exploration as well as custom visualization.\nOUAV has evolved significantly over the years, transitioning from simple reporting tools to comprehensive platforms that empower utility companies to gain real-time insights, predict future trends, and visualize complex datasets at multiple levels of granularity. Key areas of advancement include: \nReal-time data streaming: OUAV has expanded its ability to handle real-time data streams, offering utilities immediate insights into operational activities such as energy consumption, water usage, outage events, and system performance.\nPredictive and prescriptive analytics: Beyond descriptive analytics (merely what has happened), OUAV integrates predictive and prescriptive capabilities to forecast issues and prescribe optimal courses of action.\nAdvanced visualizations: From interactive dashboards to heat maps, time-series visualizations, and geographic overlays, Oracle has refined its visualization tools to handle increasingly complex data sets while making them accessible to a wide range of users.\nThis powerful evolution enables utilities to move from reactive to proactive management, allowing for more efficient operations, improved customer satisfaction, and enhanced risk mitigation. It enables intuitive, powerful visualization and reporting across digital utilities so they can: \nGain a comprehensive view of all mission-critical data and business performance \nInfuse core applications with analytics to ensure teams can make data-driven decisions\nAggregate consumption data and calculate billing determinants automatically, from simple
to complex rate structures\nLeverage data to identify and understan loss \nOracle Utilities gives utilities a multi-level approach in identifying and understanding loss. At the core of that approach is the robust meter solution, which is AMI-ready, supports customer and distribution operations, and leverages the full power of meter data to identify errors or anomalies and ensure billing performance, even as AMI programs expand. \nBeyond these core metering capabilities, Oracle Utilities Analytics visualization embedded in the solution makes it easy to identify outliers from normal usage patterns and then capture those accounts for corrective action. Going further, Oracle Utilities Analytics Insights is a cloud-based analytics solution built specifically for utilities to maximize the value of operational data. Included in this solution are modules specific to technical loss and theft detection, including among others:\n •Meter failure detection • Theft and loss detection at the meter level • Theft and
loss detection at the grid level\nDesigning Oracle Utility Analytics visualization reporting\nOracle Utilities business applications manage custom data in characteristics tables that are associated with a solution’s maintenance objects. Characteristics of the type of predefined value and ad hoc value may also be exposed in Oracle Utilities Analytics visualization subject areas so they can be used for analysis and data visualization. \nUsers can build and centrally manage reports and interactive dashboards to help ensure governance and control. They can create static reports or fully interactive parameterized dashboards with functions such as drill down and filtering. They can ensure business users access certified reports from verified data sources that leverage approved metrics, calculations, and KPIs. \nUpdate once to ensure everyone uses the correct version of the data, then schedule reports to automatically refresh and notify business users. When authorized, users can print, share, and save reports or export the base data. Integration with wider services enables users to act on information directly from the report.\nOf course, creating effective OUAV reports and dashboards requires careful design and attention to user needs. Here are some best practices for making the most out of Oracle’s visualization capabilities:\n1. Tailor dashboards to different roles—Different teams in a utility company have different priorities. Customizing dashboards to meet the specific needs of operations, customer service, finance and executive teams ensures that each group has quick access to the data that matters most.\nOperations teams: Should focus on real-time performance metrics, outage tracking, and infrastructure health.\nCustomer service teams: Should focus on customer satisfaction scores, billing anomalies, and service requests.\nExecutives: Should focus on high-level KPIs like revenue, customer engagement, and overall service delivery.\n2. Keep visualizations simple and intuitive—Even advanced analytics can become overwhelming if the visualization isn’t well designed. Keep visualizations simple, clear and focused on actionable insights. Avoid cluttering dashboards with too much data and ensure that every element serves a specific purpose.\nUse color coding to highlight issues (e.g., red for critical issues, yellow for warnings).\nProvide clear legends and tooltips to help users interpret the visuals.\n3. Leverage predictive insights for better forecasting—Use the power of predictive and prescriptive analytics to inform long-term strategies. For example, utilities can use demand forecasting to plan capacity and resources, or predictive maintenance insights to optimize asset life cycles.\n4. Ensure cross-platform compatibility—With many utility professionals working in the field, it is crucial that dashboards and visualizations are mobile-friendly. Make sure your reports are optimized for mobile devices, allowing team members in the field to receive real-time insights and make data-driven decisions.\n5. Monitor and refine continuously—Once your visualization reports are set up, it’s important to continuously monitor their effectiveness. Regularly refine and adjust dashboards to ensure they remain relevant to the changing needs of your teams and business goals.\nConclusion\nOracle Utility Analytics visualization reporting is not just about creating visually appealing charts and graphs; it’s about transforming raw data into actionable insights that drive better decision-making, improve operational efficiency, and enhance customer satisfaction. By leveraging advanced features such as real-time monitoring, predictive analytics, geospatial visualizations, and interactive dashboards, utilities can stay ahead of the curve in a rapidly evolving industry. \nWhether you’re improving grid management, enhancing customer service, or optimizing resource allocation, OUAV reporting is the key to unlocking the full potential of your utility’s data.


Advanced Oracle Utility Analytics Visualization (OUAV) Reporting: Designing Data-Driven Solutions For The Future Of Utilities

By Naveen Madishetti, HEXstream data analyst\nIn the world of data analytics, Power BI has become a prominent tool for transforming raw data into meaningful insights. One of its most powerful features is drill-through functionality, which enables users to go deeper into data points without losing context. In this blog we will explore what drill-through is, how to implement it, and then spotlight benefits of using drill-through.\nWhat is drill-through in Power BI\nDrill-through is a feature in Power BI that enables users to navigate from a high-level summary report to a detailed data view of specific data point. Unlike drilldown, which explores hierarchical data within the same visual, drill-through takes you to a separate page focused on the selected item.\nImplementation of drill-through in Power BI\nCreate a summary page \n·      In Power BI we need to create a new page and rename it as the summary page.\n·      Add a visualization that displays summarized data.\n·      Make sure the visual contains fields relevant to the analysis, such as sales, region, etc.\n\n\nCreate a detailed-view page \n·     Create a second page and rename it as detailed-view page.\n·     Add visualization that contains detailed information about a particular data point such as order ID, product, sales person and unit price.\n\n\nAdd a drill-through field\n·     Drag the field that you want to use as the drill-through filter into the drill-through section in the visualizations page. This field acts as the key that connects the summary page to the detailed-view page.\n·     Power BI automatically creates a back button that allows users to return to the main report page.\n\nHow to use drill-through functionality\n·     Go to the summary page. \n·      Right click on a data point such as region or sales field.\n·      Choose detailed view by selecting drill-through.\n·      The data on the detailed-view page will be filtered according to the value that was chosen.\n·      To reset the selected filter on the detailed view page, click the reset button. \n·      To go back to the summary page, click the back button.\n\n\n\n\nBenefits of using drill-through\nEnhanced data exploration—Drill-through enables users to navigate into specific subsets of data without overcrowding the main report, offering a focused view of the details that matter most.\nImproved user experience—Seamless navigation between summary and detailed views creates a smooth workflow.\nBetter decision-making—By enabling a dynamic approach to data exploration, drill-through
provides deeper insights that help stakeholders make more informed decisions.\nStreamlined reports—Drill-through helps keep reports clean and structured by reducing the need for excessive visuals on a single page.\nConclusion\nDrill-through is a powerful feature that bridges the gap between high-level summaries and detailed analysis by enabling users to explore data with precision and clarity. This functionality not only improves user experience by keeping reports clean and structured, but also empowers decision-makers with deeper, more actionable insights. \nBy leveraging drill-through, businesses can uncover hidden patterns and trends effectively. Implementing this functionality ensures clear and insightful reporting for better strategic outcomes.\n\nCLICK HERE TO CONNECT WITH US ABOUT WAYS DRILL-THROUGH CAN BENEFIT YOUR UTILITY.\n

What Are The Benefits Of Drill-through In Power BI?

By Satish Saini, HEXstream utilities industry specialist \nIn \nthe first installment of this series\n, we discussed that among the latest innovations, \nArtificial Intelligence (AI)\n and its applications are rapidly increasing in supporting operations and management of the energy & power system. \nAI-powered algorithms\n and \nmachine learning (ML) models\n are more and more innovated, customized and advanced to be widely used in current utility transformation and grid modernization efforts.\nIn this segment, some specific use cases of \nAI-Powered application \nin utilities\n \nusing\n Advanced Metering Infrastructure (AMI) / smart metering \ndata will be discussed.\n\nUtility Supply & Demand, Load Forecasting & Planning, Energy Losses\nUtility Demand and Load Forecasting: AI can provide better insights about load and demand patterns across the power grid and help with future demand forecasting by analyzing historical and real time demand and consumption patterns, weather data, and other factors for various communities and segments of the power grid. This can help utilities to better plan their assets upgrades and grid expansions based on forecasted demand.\nUse Case:\nDemand and Load Forecasting\nGrid Expansion Planning\nPeak Load Management\nGrid Supply & Demand Balancing - AI can help utilities in balancing supply and demand by analyzing available supply from grid resources and predicting demand and consumption across the grid and help effectively manage energy distribution. This improves the grid’s reliability by reducing the likelihood of blackouts, especially during peak load times. Combining with further utility initiatives and programs on Demand Response (DR) / Demand Side management (DSM), this can be of big relief to utilities when the grid is constrained. Utilities can use AI driven communication to issue alerts to customers on incentives for participating in demand response programs, and subsequently manage demand.\nUse Case:\n·       Supply & Demand Balance\n·       Demand Response (DR) and Demand Side Management (DSM)\nEnergy Losses, Consumption Anomalies, and Fraud Detection- By analyzing smart metering data, AI models can detect unusual patterns or anomalies in energy consumption for specific consumers, communities and grid segments, such as significant spikes or drops in usage. This can help identify potential problems like faulty meters, power theft / pilferage, frauds, or service lines / equipment malfunctions. These AI algorithms on energy losses by spotting suspicious patterns can help prevent revenue losses and ensuring the accuracy of billing by taking corrective actions.\nUse Case:\nFrauds & Energy Losses detection\nFaulty Meter Detection\nConsumption Anomalies & Billing Errors\nUtility Assets Management and Performance Insights\nAssets Loading: Using and analyzing the historical and real-time AMI data on energy consumption and rolling up to upstream grid assets, AI-powered algorithms can help utilities with meaningful insights on assets loading conditions (like transformers and feeders) and issue alerts on abnormal loading / anomalies. This can help monitor critical assets load and predict future overloading situations, helping utilities take proactive steps in load shifting or assets upgrades.\nUse Case:\nAssets Loading Analysis, Overloading alerts, Upgrade Forecasting\nPower Quality: AMI data on power quality parameters like voltage, active and reactive power, power factor, frequency and others can provide useful information and prediction on key operational and performance parameters of critical grid assets. Relevant alerts can be issued triggered by anomalies, significant spikes and dips which can help utilities plan
proactive, preventive and corrective inspections and maintenance activities. This can help prevent significant failures, expensive breakdowns and downtime and saving operational costs\nUse Case:\nAssets Power Quality monitoring\nAssets Performance insights\nAssets Predictive & Condition-based Maintenance\nCustomer Services, Experience and Engagement:\nEnhanced Customer Experience (CX/CIS) with Dynamic Pricing and Energy Efficiency – Using the historical and real-time metering data, AI-driven algorithms can provide energy consumers with insights on their energy usage patterns and can suggest energy-saving tips by adjusting consumption, taking energy efficiency measures and leading to costs savings. This offers a big benefit when the utility has Dynamic Price structure / Time-of-Use rates. Customer can receive alerts from AI driven communication on energy efficiency and incentives for participating in demand response programs as offered by the utility. It can suggest optimal times for high-energy-consuming tasks (like heating or cooling) with minimum applicable rates (wherever applicable).\nUse Case:\nEnergy Management, Conservation & Efficiency\nDynamic Pricing, Time-of-Use rates and Load Management\nConsumer Engagement and Energy Management – By analyzing individual customer load and energy consumption data, AI systems can provide personalized feedback and recommendations to customers for efficiently managing energy at their premises. This can be achieved through AI driven alerts and communication on energy usage, saving tips, financial and cost saving benefits and feedback.\nUse Case:\nCustomer Engagement and Cost savings\nIntegration with Building and Home Automation System and IoT Devices - AI can integrate AMI data along with data from IoT (Internet of Things) devices deployed at consumer premises, such as smart appliances, thermostats, and sensors and make relevant and efficient forecasts and predictions. This allows for even greater optimization of energy usage, as AI can control devices in real-time to optimize energy consumption and efficiency.\nUse Case:\nAMI Integration with Home Automation and IoT\nTo conclude, utility AMI data can provide a lot more valuable insights, forecasts and predictions regarding load and demand, assets performance, customer services and energy management. All this can help utilitites toward efficient transformation and grid modernization initiatives.\n\nContact us for further details as to how artificial intelligence can help your utility to effectively manage the grid, critical assets and enhancing customer experiences.\n\nUp next: We will discuss more about further use cases for T&D utilities in relation to Advanced Distribution Management System (ADMS) towards grid modernization and utility transformation. \n\n


AI-Powered Use Cases In Utility Transformation And Grid Modernization

By Swathi Ambati, HEXstream data solutions engineer\n \nIn a \nprevious article\n, we explored how Oracle APEX can use an Excel spreadsheet to quickly create an application that enables data entry, reporting and dashboard functionality. \nHowever, we are in the AI era where artificial intelligence can further accelerate development.\nThis article will focus on the AI features available in APEX, how to configure them, and includes a short demo on creating an application based on existing database tables.\nAI-powered capabilities in APEX\nOracle APEX offers several AI-powered features to improve development efficiency, including:\n·      AI-assisted features in APEX\n·      Integration with OpenAI Assistant via Dynamic Actions\n·      Oracle AI services: Vision, Oracle Digital Assistant (ODA), Document Understanding, Generative AI, and Select AI\n·      Retrieval Augmented Generation (RAG) capabilities in AI-supported applications with APEX\nOracle APEX simplifies AI integration by abstracting popular AI providers such as OpenAI and Cohere. This is achieved through a REST infrastructure where authentication is managed via web credentials. The AI service interacts seamlessly with the Oracle database and APEX, while relying on AI providers for backend inferencing.\nUse cases of AI in APEX\n1. Prompt-based app creation—Developers can create APEX applications using natural language descriptions. This significantly reduces development time and improves
accessibility.\n2. Natural language to SQL (NL2SQL)—Users can enter queries in natural language, which are automatically converted into SQL statements. This simplifies the creation of reports and interaction with the database, especially for non-SQL users.\n3. AI assistant for real-time support—An AI-powered chat widget can assist users with app navigation, query creation and troubleshooting.\nLet's look at a simple example of how AI-powered functions can create an application based on existing data.\nActivate AI services in APEX\nConfigure the generative AI services in APEX by setting up the AI provider (e.g. OpenAI, Cohere).\nTo leverage AI features in APEX 24.1, you first need to configure a Generative AI service within your workspace. This can be done declaratively in just a few steps.\nStep-by-Step Guide to Configuring a Gen AI Service\nNavigate to the Generative AI Settings\nFrom your APEX workspace homepage, go to App Builder.\n\nClick on Workspace Utilities.\n\nSelect Generative AI from the available options.\n\nCreate a New Generative AI Service\nClick Create to set up a new AI service.\n\nChoose an AI provider. APEX 24.1 supports:\nOpenAI\nOCI Generative AI Service\nCohere\nExample: Configuring OpenAI as the AI Provider\nSelect OpenAI from the provider list.\nEnter a service name for easy identification.\n(Optional) Provide a static ID, which will be referenced when using the APEX_AI package.\nEnable the toggle button for App Builder access, allowing you to use the AI service within your applications.\nSet Up API Credentials\nThe base URL is auto-generated—leave it as default.\nEnter your OpenAI API key. (If you don’t have one, visit \nplatform.openai.com\n to sign up for an API key.)\nThe default AI model is set to GPT-4.0.\nFinalize the Configuration\nClick Create to save your AI service configuration.\n\nYour Generative AI service is now ready to use APEX!\nEnter a query in natural language:\n\nUse the APEX wizard to describe the application in natural language.\nThe AI generates a functional APEX application with corresponding tables, forms and reports.\nWe can see that in the demo video the AI Assistant generates a basic application as a starting point based on a two-line request. You can customize the application as needed, and the APEX AI Assistant can assist in creating and modifying SQL queries throughout the customization process.\nThis video demonstrates the process detailed above:\n\nConclusion\nAI-powered functions in Oracle APEX significantly speed up development by automating SQL generation, app creation and chatbot integration. With AI-driven features, developers can focus on building smarter applications faster and more efficiently.\n\nCLICK HERE TO CONNECT WITH HEXSTREAM TO LEARN HOW TO ACCELERATE YOUR APEX DEVELOPMENT. \n

Accelerating APEX Development With AI-driven Functions

By Satish Saini, HEXstream utility industry specialist \nThere are many factors that propel utilities to adopt smarter asset-managmenet strategies informed by performance insights from relevant analytics. Consider just some of these factors: \nAging T&D grid infrastructure leading to equipment failures\nClimate change causing adverse weather events and impacting power infrastructure reliability\nCustomer needs for reliable power\nUtilities' needs for sustainable, efficient and cost-effective business operations\nRegulatory pressure to maintain operational and safety standards\nRenewable energy sources penetration to the T&D grid\n\nThere are many challenges, unfortunately, but solutions to overcome these challenges get smarter every day. Let's explore...\nAging infrastructure, grid reliability and resiliency:\n The majority of domestic T&D infrastructure is already aging and reaching the end of life, which is leading to frequent failures, blackouts, power-quality issues and increased maintenance costs. Extreme weather events and natural disasters pose further risks to grid reliability and stability and cause heavy expenses due to damages.\nAccording to the US Department of Energy (DOE), much of the US electric grid was built in the 1960s and 1970s, and 70% of the infrastructure is approaching the end of their typical 50–80-year lifecycle. This has major consequences on our communities—power
outages, susceptibility to cyberattacks, emergencies caused by faulty grid infrastructure. \nIts very hard to replace and upgrade all of this infrastructure, as numerous assets along high-voltage transmission lines and substations are quite expensive and involve huge investments.\nHowever, effective asset-management plans and automated performance insights\n \ncan help defer some of the replacement costs through predictive analysis regarding asset-health conditions and through critical condition-based monitoring and maintenance
of critical and expensive assets (think transformers, circuit breakers, etc.). \nThese insights help maximize the lifecycle of assets.\nAsset operational efficiency & performance affecting utility financial sustainability:\n Inefficient operation and performance of assets due to aging and other factors leads to higher system losses, grid/asset failures, voltage fluctuations, power-quality issues and system overloads. These things derail utility operational efficiency, of course, and hurt financial sustainability and cost recovery.\nHuge capital investment on replacement and increased operations & maintenance (O&M) costs of inefficient assets push utilities toward developing and implementing asset-management strategies to efficiently manage the grid through strategic investments, optimizing O&M costs, enhancing equipment lifecycles and improving financial sustainability.\nDigital transformation, real-time monitoring and AI-driven, automated asset-performance insights inform effective data-driven decision-making, as traditional manual inspections and maintenance methodologies lead to delays and inefficient decisions. And as you know, data-driven analytics on proactive and predictive condition-based maintenance more efficient than reactive maintenance.\nRegulatory compliance & safety standards:\n There is a greater public dependency on power. There is increased customer expectation on power quality and reliability. There are more complex requirements to comply with myriad standards and government regulations related to reliability, safety and environmental targets. Failure to meet these goals results in heavy penalties for utilities.  \nA critical part of the strategy to meet these goals is effective asset-management strategies and procedures, which help utilities ensure compliance with grid codes and safety standards (like NERC, FERC, IEEE). Further monitoring of emission levels from grid operations (substations and other assets) helps monitor and meet environmental regulations. Technologies and solutions like real-time monitoring and asset-performance insights help reduce the risks of equipment failures and outages through proactive actions. Automated analytics and reporting mechanisms enhance accuracy in compliance and reducing errors and operational costs. Win win win. \nRenewable energy resources and EV penetration impacting grid assets:\n Increasing penetration of Distributed Energy Resources (DERs) and electric-vehicle charging infrastructure at various segments along the T&D grid is creating new operational challenges for the grid due to reverse and two-way power flow and increased fault levels. This affects a large number of grid assets at transmission lines, substation and distribution feeders, all while triggering the need for heavy replacements and upgrades.\nFor efficient integration and utilization of DERs and the efficient operation of EV-charging infrastructure, efficient and optimized asset-management strategies and investment prioritization is required to maintain operational efficiency and peak performance of the grid.\nSo how do we do this? \nT&D utilities must develop and implement strong asset =-management strategies, programs and frameworks using digital technologies to ensure grid reliability, support renewables, comply with regulations, and enhance financial sustainability. A modern asset-management plan using digital technologies and data-driven, predictive, and proactive asset-performance
monitoring enables utilities to future-proof their infrastructure and improve customer service.\n\nWant more? Click here to contact us for details on asset-management strategies and asset-performance insights.\n\nNext up: A closer look at key assets along the T&D grid, pillars, strategies and roadmap for effective utility-asset management.\n


Creating Smarter Asset-Management Strategies For T&D Utilities

By Bala Sai Tilak.Thalla, HEXstream data integrations engineer\nOracle Integration Cloud (OIC) is a powerful Platform as a Service (PaaS) solution that simplifies applications and data integration across cloud and on-premises environments. It seamlessly connects homogeneous and heterogeneous applications, enabling organizations to streamline business processes and improve efficiency. \nHowever, organizations often face challenges in bridging gaps between disparate systems while ensuring scalability, security and performance. These challenges can be overcome. \nTo build a robust integration, it is essential to focus on three key aspects:\nSource system—The system from which data originates.\nTarget system—The destination where the data is delivered.\nMiddleware platform—OIC, which facilitates the integration process.\nKey considerations for effective OIC integrations\n1. Establishing secure connectivity with the connectivity agent\nMany enterprise applications still rely on on-premises systems and legacy applications. To securely integrate these with cloud-based applications, Oracle Connectivity Agent is used. This agent establishes a secure bridge between cloud and on-premises applications without
exposing internal systems directly to the internet.\n2. Handling large data volumes efficiently\nWhen integrating applications that handle bulk data, performance bottlenecks can arise. To optimize performance, enterprises should evaluate the system's processing capacity before integrating bulk data, then monitor throughput and response times to detect slowdowns.\n3. Optimizing performance with batch processing\nBulk data transfers can harm system performance. Instead of pushing large data loads at once, leverage batch processing by:\nSegmenting data internally within OIC.\nProcessing data in manageable batches instead of overwhelming the system.\nEnhancing efficiency by ensuring smooth data ingestion.\n4. Improving scalability with asynchronous integrations\nSynchronous integrations wait for an immediate response, which can result in failures if the response isn't received within 300 seconds. Instead, using asynchronous integrations (such as decouple processes) reduces system dependencies.\n5. Leveraging parallel processing for faster execution\nWith OIC Gen 3, parallel processing is now available, enabling actions to be executed simultaneously instead of sequentially. This reduces processing time by executing multiple tasks concurrently and ensures faster response times for the target system.\n6. Enhancing efficiency with scheduled integrations\nScheduling integrations is a best practice for automating routine data transfers. Use child integrations to consume and store data internally before triggering the scheduled integration;  this method ensures controlled execution and reduces system overload.\n7. Implementing a robust error-handling mechanism\nError handling is a crucial component of any integration architecture. Consider: \nRetry mechanisms: Automatically reattempt failed integration executions.\nNotification actions: Send error details via email or logs for prompt resolution.\nGlobal fault handlers: Define standardized error-handling logic across integrations.\n8. Managing changes with versioning\nDirectly modifying active integrations can lead to failures. Instead, adopt versioning to create a new version of an integration without deactivating the existing one. Allow rollback in case of unexpected failures.\n9. Ensuring security and compliance\nFor organizations that handle sensitive data, security is a top priority. Consider: \nAuthentication mechanisms: Use OAuth for APIs to secure access.\nEncrypted communication: Implement SASL PLAIN over SSL for secure message-transmission.\nBuilding a scalable, secure and efficient integration in Oracle Integration Cloud requires careful planning and adherence to best practices. By leveraging batch processing, asynchronous execution, parallel processing, scheduled integrations, error handling,
versioning, and security measures, organizations can ensure high-performance and resilient integrations.\nFollowing these methods will enable businesses to seamlessly integrate applications, improve data flow, and drive operational excellence in the ever-evolving digital landscape.\n\nCLICK HERE TO CONNECT WITH US ABOUT BUILDING SCALABLE INTEGRATIONS AT YOUR ENTERPRISE. \n\n


Tips For Building Scalable Integrations In Oracle Integration Cloud (OIC)

By Satish Saini, HEXstream utilities industry specialist \nEarlier this month, US Secretary of Energy Chris Wright and few other senators participated in an artificial intelligence (AI) collaboration session involving Department of Energy (DOE) scientists and other professionals from AI technology field. They discussed the plans for the next four years to create America’s “New Golden Age,” win the AI race, and make US energy dominant again, noting that energy is essential to the nation’s security, the well-being of our citizens, and lives of people around the world.\nIn October 2024 last year, the DOE published “Artificial Intelligence and Transforming the Energy Landscape in US,” which detailed plans for applying AI to address key challenges across the energy sector through following key initiatives:\nModernizing the aging and complex electricity grid using AI-powered tools for enhancing grid resiliency, optimizing operations, efficient integration of renewable energy to the grid and improving load-forecasting and other operational data insights.\nAccelerating energy innovation through the development of new energy technologies in power generation and storage to meet energy goals, while using AI to automate processes and improving assets' lifecycle performance.\nImproving energy efficiency through AI-driven design, solutions and control systems for buildings, transportation, industrial processes and equipment through optimizing energy consumption.\n\nOur modern, complex energy and power system is triggering the need for automated tools and technologies with the rising electricity demand. It requires more penetration of distributed, renewable energy sources across various segments of the transmission and distribution grid, in reaction to the rising number of grid-connected devices stemming from the adoption of smart-grid technologies and behind-the-meter (BTM) systems like solar energy, storage and EV charging systems.\nFurther, climate changes and adverse-weather events are severely impacting the power-grid infrastructure, triggering the need for a more reliable and resilient grid. Operating this complex, vast and widely scattered grid with millions of devices and assets pose a big challenge for the utilities. You already know this; think planning activities for the complex grid, forecasting for load, predictive analysis and insights for assets performance and maintenance needs for these innovative tools and technologies. Also consider how regulatory compliance, safety standards, utility-business operations and sustainability requirements need efficient data-exchange and decision-making tools through the integration of various silos and platforms within the utility.\nSupporting all this in the energy and power system, AI and its applications are rapidly increasing in quick, accurate and automated data analysis and reporting. This is a good thing. AI algorithms and machine learning (ML) models are increasingly customized and advanced for power-system-specific applications and uses. These are becoming more effective and widely used in current utility transformation and grid-modernization efforts by utilities. \nIn short...they have to be. \nA massive amount of data is generated with the rising deployment of Distributed Energy Sources (DERs), Advanced Metering Infrastructure (AMI), Advanced Distribution Management System (ADMS), Smart Grid, Asset Performance Management (APM) and other technology platforms within T&D grids. This huge data from multiple platforms needs to be quickly and effectively integrated, processed and analyzed to help utilities in efficient decision making in operation and maintenance activities. \nManaging this much volume and integration cannot be done through manual processes. The use of AI and ML is providing a huge amount of support to utilities through advanced algorithms and machine-learning models.\nA report by International Energy Agency (IEA), in reference to an analysis from Indigo Advisory Group, suggests that AI is already serving more than 50 different uses in the energy system, from grid operation, assets management, load forecasting and others. The report further mentions that the market for AI in the energy sector could be worth up to USD $13 billion in the coming period. \nIncentives like that spur transformation and modernization efforts. \n\nContact us for further details as to how artificial intelligence can help your utility to effectively manage your critical assets and enhance your customers' experiences.\n\nUp next: Specific use cases for T&D utilities in relation to asset-performance insights, load and demand planning/forecasting, grid modernization and utility transformation. Stay tuned! 

The Role Of Artificial Intelligence (AI) In Utility Transformation And Grid Modernization

By Bharath Moola, HEXstream data solutions engineer\nIn today's data-driven world, industries like utilities, telecoms and service providers rely on analytics for operational efficiency, customer satisfaction, and decision-making. This you already know. One powerful tool for interactive data analysis is Power BI Q&A, which enables users to ask questions in natural language and receive visualized insights. This might be new to you. \nThis blog explores how to train Power BI Q&A for better results, using outage analytics as an example. \nWhat is Power BI Q&A? \nPower BI Q&A enables users to query data in plain English, generating charts, tables and graphs without complex DAX formulas. For instance, asking “What were sales last quarter?” returns relevant data instantly. However, to improve accuracy, especially with domain-specific data like outages, you need to train the Q&A model. \nExample: Outage Analytics Training \nImagine an outage dataset with control zones labeled as Levels. Power BI may not recognize this as a region or county, so training involves adding synonyms: 
Level1_name – Company, Level2_Name – Region, Level3_Name – Division, Level4_Name – County and Level5_Name – Substation\n\nWith these synonyms, a user could ask: "How many Outages occurred in Canton County in 2017?"\n\nOr "Show me all the outages by company in 2015."\n\nSimilarly, when configuring your data model, be sure to add synonyms (where relevant) for fields that have multiple references within your organization. For example, "Outage Data Set" might have multiple dates, but users want to filter by year or other time references, which is “Event Start Date,” so you may need to add synonyms. Power BI Q&A understands different ways of asking about dates.\nAdded Synonyms:\nEvent Start Date → \nYear, Start Date, Occurrence Date\n\nTips for Better Q&A Results\nUse Clear Field Names—Give your fields descriptive labels so Power BI Q&A can easily map user queries to the data.\nAdd More Synonyms Over Time—Monitor how users phrase questions; if Q&A struggles, add new synonyms that match how your team speaks.\nProvide Sample Questions—In Power BI Desktop, you can insert sample questions that guide users and improve Q&A accuracy.\nSet Up Default Aggregations—Assign default aggregations (e.g., sum, average) for numerical fields to ensure accurate calculations. \nEnable Suggestions—Allow Power BI to suggest questions, helping users frame queries correctly. \n\nTraining Power BI Q&A with synonyms and context-specific terms enhances its ability to deliver accurate insights. Regular updates ensure seamless, user-friendly data analysis, making it an invaluable tool for any industry. \n\nLEARN HOW OUR TEAM CAN OPTIMIZE YOUR POWER BI EFFORTS. CLICK HERE TO CONTACT WITH US.\n

Power BI Q&A: Training For Better Data Insights

By John Apostolo, HEXstream application developer \n \nModern AI models face a paradox: achieving state-of-the-art performance requires immense computational power, yet the cost of scaling remains prohibitive. While dense architectures have traditionally set the standard for reliability and performance, they suffer from inefficiency at scale. Meanwhile as a counterbalance, Mixture-of-Experts (MoE) offers a promising path toward higher efficiency but introduces complexity in model routing. \nIn this post, we explore how a dual approach—refining dense models while leveraging MoE—enables AI to break through scalability limitations while maintaining performance and efficiency. We also examine the significance of high-quality pre-training and future-proofing techniques for long-context and multimodal AI. \nHigh-quality pretraining on curated data–the foundation of AI reasoning\n \nThere is a problem with unfiltered data.  \nMany open-source models rely on massive datasets scraped indiscriminately from the internet. While sheer volume can improve knowledge coverage, it also introduces noise, redundancy, and factual inaccuracies—leading to models that struggle with logical consistency and deep reasoning. \nRather than prioritizing scale over quality, a high-performing AI model should be trained on high-quality, diverse, and well-structured sources such as books, academic papers, and formally vetted content. Filtering out low-value internet chatter minimizes hallucinations and ensures the model learns concepts over memorization. \nWhy this matters\n \nHigher factual accuracy—reduces misinformation propagation. \nBetter logical reasoning—moves beyond surface-level pattern recognition. \nEnhanced multilingual support—ensures robust understanding across languages. \nRedefining dense model efficiency: smarter scaling over larger models\n \nThe problem: compute-heavy, unsustainable growth \nDense models, by design, activate all parameters for every token processed. While this guarantees consistency and predictability, it also leads to exponential compute costs as models scale. \nOptimized dense model design\n \nA breakthrough in dense architecture isn’t just about adding more layers or parameters—it’s about refining efficiency per token. By carefully optimizing: \nNetwork depth and width for better weight utilization. \nTokenization strategies to reduce redundant processing. \nMemory-efficient scaling techniques that allow for high performance at lower compute costs. \nThe outcome: new standards in dense AI \nThis approach achieves state-of-the-art coherence and reasoning without requiring the extreme sparsity of MoE, proving that dense architectures can remain viable when optimized correctly. \nMixture of experts (MoE): the key to unlocking scalable AI\n \nThe problem: scaling beyond dense models \nAs AI models surpass billions of parameters, activating every neuron for every token becomes impractical—leading to massive compute costs and energy inefficiencies. \nSo how does MoE solve this? MoE introduces a novel approach where only a fraction of the model’s parameters are used at a time, thanks to: \nSpecialized routing mechanisms that direct inputs to the most relevant “experts.” \nDynamic sparsity, allowing the model to selectively activate only the necessary computational pathways instead of the full network. \nAnd what is the impact of MoE? By adopting this selective activation strategy, models can scale beyond dense architectures without a linear increase in compute cost. This results in: 
✔ Greater model capacity—They can handle more complex reasoning tasks. 
✔ Lower computational demands—Reduces cost per inference. 
✔ Specialization without redundancy—Experts specialize in different aspects of knowledge, improving response accuracy. \nArchitectural synergy: when to use dense vs. MoE\n \nThe tradeoff: reliability vs. efficiency \nOrganizations developing AI solutions typically face a crucial tradeoff: \nDense models provide predictable, uniform activation—ideal for tasks where reliability, interpretability, and deterministic responses are necessary. \nMoE models are optimized for efficiency and scalability—better suited for large-scale inference where compute costs are a primary concern. \nThe benefit of offering both architectures\n \nBy developing both dense and MoE models, researchers and organizations gain: \nFlexibility to choose based on task complexity and computational resources. \nOptimal performance scaling—dense for high-precision tasks, MoE for high-scale efficiency. \nThe best of both worlds—reliable, full-parameter models alongside scalable, cost-effective MoE systems. \nFuture-proofing AI: long-context and multimodal capabilities\n \nThe challenge: memory limitations and cross-modal understanding \nMost AI models struggle with: \nLong-context retention—forgetting key details in extended interactions. \nMultimodal processing—seamlessly integrating text, image, and video understanding. \nTo create an AI model that is future-ready, advancements must include: 
✔ Extended context length support—ensuring information retention across long-form content. 
✔ Multimodal alignment techniques—integrating multiple data types for a cohesive understanding of text, images, and beyond. \nOutcome: versatile AI-ready for complex applications\n \nThe combination of: \nDense model precision \nMoE scalability \nMultimodal readiness \n…ensures AI is prepared for enterprise-scale applications, research, and next-generation problem-solving. \nWhy this dual approach is the future\n \nThe debate between dense vs. MoE is not about one replacing the other, but rather about how they complement each other in modern AI development. By: \nImproving dense-model efficiency \nLeveraging MoE for scalable reasoning \nIntegrating long-context & multimodal readiness \n…AI models can deliver superior performance while maintaining computational efficiency. \nThis dual-track approach ensures AI remains adaptable for both research and enterprise applications, future-proofing AI capabilities for years to come. \nWANT MORE? PART 2 OF THIS BLOG WILL SPOTLIGHT STRATEGIES TO REALIZE THES GAINS. STAY TUNED…AND \nCLICK HERE\n TO CONNECT WITH US TO LEARN HOW YOUR UTILITY CAN BRIDGE AI SCALABILITY AND EFFICIENCY. \n 

Bridging AI Scalability And Efficiency: The Synergy of Dense and Mixture-of-Experts Architectures

By Bharath Moola, HEXstream Data Solutions Engineer\n\nOrganizations leveraging Oracle Analytics Cloud (OAC) often seek to integrate with Microsoft Power BI for enhanced visualization and self-service analytics. This blog outlines efficient ways to connect these platforms for seamless data analysis.\nUsing RPD in Power BI\nOracle Analytics Cloud utilizes the Repository Database (RPD) as a metadata layer to structure and manage data models. RPD plays a critical role in integrating OAC with Power BI by defining the logical relationships between data sources and ensuring seamless connectivity. By leveraging RPD, users can create subject areas, define calculations, and
establish security constraints before exposing the data to Power BI.\nConnecting Power BI to an RPD-enabled data source requires setting up an Oracle BI Server ODBC connection. Once configured, Power BI can access subject areas and logical tables designed in the RPD. This approach ensures that the predefined business logic and calculations embedded in the RPD are retained, providing a consistent analytical experience across both platforms. Additionally, using RPD allows organizations to manage role-based
security and data access policies centrally, ensuring compliance with governance standards.\nTo integrate the RPD with Power BI:\nConfigure Oracle BI ODBC drivers and set up connection details within Power BI.\nFetch structured data from OAC’s logical model.\nBuild Power BI reports without modifying the source structure.\nThis method is especially useful for organizations managing large and complex datasets, ensuring that data remains structured and governed efficiently.\nPrerequisites\nMicrosoft Power BI Desktop must be up to date:\n\nOracle Analytics Cloud should be running:\n\nDownload the latest version of the Oracle Analytics Power BI Connector:\n\n \n\n
Place the latest MEZ file in the following location for your user: \n( c: users/<username>/Power Bi desktop/customconnections):\n\nSteps to Connect Oracle Analytics Cloud to Power BI\nIn Power BI Desktop: Click on Options and Settings:\n\nUnder GLOBAL > Security: Select (Not Recommended) Allow any extension to load without validation or warning:\n\nAdjust memory settings:\nUnder GLOBAL > Data Load, set the value for Maximum memory used per
      simultaneous evaluation (MB) based on your machine’s available memory:\n\nObtain Power BI API Key from OAC\nLog in to OAC > Navigate to User Profile > Copy the Power BI API Key:\n\nLog in to Power BI desktop > Get Data > Search for Oracle Analytics >Select Oracle analytics (Beta) > Paste the API Key in Power BI when prompted:\n\nAuthenticate with OAC and complete the login process:\n\nClick Connect.\nAccessing Subject Areas in Power BI\nOnce connected, the screen below will be displayed:\n\nExample: Power BI Dashboard Built Using OAC Subject Areas\n\nIntegrating OAC with Power BI using RPD ensures a seamless transition of pre-modeled data while maintaining governance, security, and analytical consistency. By leveraging OAC’s logical layer, professionals can extract meaningful insights without compromising data integrity. With the right configurations, Power BI serves as a powerful front-end visualization tool while OAC manages backend data processing and modeling.\n\nLEARN HOW OUR TEAM CAN OPTIMIZE YOUR POWER BI EFFORTS. CLICK HERE TO CONTACT WITH US.\n


A Quick Guide: Integrating Oracle Analytics Cloud With Power BI

By Devika Gayapu, HEXstream data-solutions engineer\nThe Oracle Business Intelligence Cloud Connector (BICC) is a specialized tool designed for efficiently extracting large volumes of business-intelligence data and related content from Oracle applications. Built to handle the heavy lifting of transferring vast amounts of data, Oracle BICC ensures that no data is lost during the process. It also offers robust security by adhering to the security standards of Oracle Applications Cloud.\nOracle BICC can manage large data volumes while supporting both full and incremental data extracts. It uses data stores to facilitate the movement of data. In simple terms, data stores act as intermediary storage areas, enabling staging, transformation, and finally loading of data to its destination, while maintaining data integrity and consistency throughout the process. By utilizing BICC Data Stores, organizations can automate the extraction process, scheduling both one-time and incremental extracts based on needs.\nOracle BICC can extract data from various Oracle products, such as Oracle Fusion Applications (including Oracle CX, Oracle HCM, Oracle ERP and Oracle SCM), and transfer it to external storage. These extracts can be either complete or partial and can be scheduled or run on-demand to ensure that data remains up-to-date without requiring manual intervention. BICC also supports incremental extracts, which capture only the data that has changed
since the last extraction. Additionally, Oracle BICC can monitor data extracts and check logs for updates.\nOracle BICC utilizes both SOAP and REST APIs for data extraction and integration. SOAP API is primarily used for scheduling and triggering BICC extracts, whether they are full or incremental, while the REST API offers more flexibility for real-time data integration. REST APIs are data-driven, enabling access to resources via URIs. SOAP, on the other hand, is more structured and focused on specific functionalities. \nWhile REST APIs are preferred for better performance and scalability—particularly for low-volume, real-time transactions or when integrating with external systems—SOAP is generally used for managing tasks like scheduling BICC extract jobs. REST is favored for direct data access and updates.\nOverall, BICC provides a streamlined and optimized approach to syncing data, making it easier for businesses to maintain accurate, up-to-date data for reporting and decision-making. \nAnd while Oracle BICC is currently used to extract data from Oracle Cloud applications like Oracle CX, Oracle HCM, Oracle ERP and Oracle SCM, in the future Oracle plans to utilize it in other functional areas, such as Oracle EWDI applications.\nWe’ll keep you posted. \n\nCLICK HERE TO CONNECT WITH US TO LEARN HOW BICC BEEN BENEFIT YOUR UTILITY.\n \n


Supercharged Data Extraction And Integration With Oracle BICC

By Swathi Ambati, HEXstream data solutions engineer\n \n“We’re now building a lot of our applications in APEX. And our productivity gains are, again, a factor of 10. And we build the applications in one tenth the time, or one tenth the amount of people, or at one tenth the cost. But these are not typical low-code applications. These are applications that can scale to millions of users and all over the world.”  Larry Ellison, Oracle founder and CTO \nIf you’ve heard of Oracle APEX but aren’t quite sure what it is, what it does, or how and where you can use it, you're in the right place. This article will provide clear, straightforward answers to all your questions. \nThe ultimate low-code platform for web applications\n \nOracle Application Express (APEX) is a low-code development platform that enables users to build scalable, secure and data-driven web applications using only a web browser. APEX is fully integrated with the Oracle database and enables both developers and business users to create applications with minimal coding, making it an ideal solution for rapid application development. \nKey features of Oracle APEX\n \nSeamless integration with the Oracle database. \nNative Oracle database support—APEX runs directly in the Oracle database and ensures high performance. \nSQL, PL/SQL and REST API support—developers can easily access and manipulate data. \nAutomatic data synchronization and JSON support—facilitates real-time updates and seamless integration with external sources. \nEnterprise-class security\n \nRobust authentication and authorization controls—supports various authentication methods, including Single Sign-On (SSO) and OAuth 2.0. \nIntegrated protection against SQL Injection & Cross-Site Scripting (XSS)—ensures secure application development without additional configurations. \nComprehensive auditing and session management—helps monitor and track application activity. \nPerformance\n \nOptimized for high performance—runs natively within the Oracle database and eliminates the need for middleware. \nHandles thousands of concurrent users—ideal for high-volume enterprise applications. \nSupports cloud and on-premises deployment—easily deploy applications in the Oracle Cloud or in an on-premises environment. \nHow does Oracle APEX work\n? \nOracle APEX follows a simple three-tier architecture that ensures efficient and fast application execution: \nWeb browser: Users interact with the APEX application through a browser. \nOracle REST Data Services (ORDS): ORDS acts as a bridge and forwards requests from the browser to the Oracle database. \nOracle database: The entire business logic, data-processing and rendering takes place in the database. The response is then sent back to the browser. \nThis architecture offers latency-free data access and ensures maximum performance and scalability for APEX applications. \nCommon use cases for Oracle APEX\n \nAPEX is used across industries to create modern, data-driven applications. Here are some of the most common use cases: \nBusiness applications: Internal tools, HR portals and CRM (Customer Relationship Management) applications. \nData reports and dashboards: Easy creation of interactive reports and real-time analytics. \nWorkflow automation: Automate business processes with APEX processes and dynamic actions. Modernizes legacy systems by converting outdated Excel spreadsheets, forms and reports into web-based applications. \nPublicly accessible applications: Develops self-service portals for customers, vendors and suppliers. \nCloud and hybrid deployments: Deploys applications in the Oracle Cloud, on-premises or in hybrid environments. \nQuick demo:\n Create an APEX application in minutes! Oracle APEX makes application development incredibly fast. With just one simple data set you can upload data, define relationships and configure UI components in minutes. Take a look: \n\nThis is a quick and efficient way to create an app from a .csv file. Not only does it generate the app, but it also creates the necessary database table to store the data. I downloaded a dataset from \nData.gov\n, and within minutes, I had a comprehensive overview of elevator and escalator outages across all NYC stations over the years, along with a detailed report featuring advanced search and filtering options. \nWant to see a step-by-step tutorial on how to create an APEX application? Stay tuned for the next article, where we explore the power of APEX + AI! And \nclick here\n to connect with us about a Proof of Concept (PoC) tailored to your requirements using Oracle APEX!  

Ten Times Faster! Ten Times Cheaper! New Wins With Oracle APEX

By Ashwin Kota, HEXstream integrations and analytics lead \nAn execution plan is essentially a roadmap that the Oracle Database follows to execute a SQL statement. It outlines every step, from how the data will be accessed to the order of operations and resources required. Execution plans are generated by the Oracle Optimizer, which carefully evaluates the SQL statement to find the most efficient path for retrieving data.\nUnderstanding execution plans is crucial because they can significantly improve database performance, guide effective resource management, and help troubleshoot slow queries.\nHow to Generate an Execution Plan\nTo start analyzing a query, you’ll need to generate its execution plan. Here’s how to do it in Oracle SQL:\nWrite the SQL Query: Begin with the SQL statement you want to analyze.\nUse the EXPLAIN PLAN Command: Before your SQL statement, add the EXPLAIN PLAN FOR command. For example:\nsql\nCopy code\nEXPLAIN PLAN FOR SELECT * FROM employees WHERE department_id = 10;\nDisplay the Execution Plan: After running the EXPLAIN PLAN command, you can view the execution plan by running:\nsql\nCopy code\nSELECT * FROM TABLE(DBMS_XPLAN.DISPLAY());\nFor those who prefer visual aids, tools like Oracle SQL Developer and Toad for Oracle provide graphical interfaces to view and analyze execution plans with useful visual metrics.\nKey Components of an Execution Plan\nAn execution plan consists of various components that give insights into how a query will be executed. Here are some key terms:\nCost: A numeric indicator estimating the resources required for the query. Lower costs typically suggest a more efficient plan, though the cost alone isn’t always the best metric for performance.\nRows: This is an estimate of the number of rows processed at each step. If the estimate is high, it could indicate a performance issue, especially if fewer rows are needed.\nOperation: The actions Oracle will perform, including:\nTABLE ACCESS FULL: Reads all rows in a table. While common, full table scans can be costly for large tables.\nINDEX RANGE SCAN: Uses an index to retrieve a range of rows, usually faster than a full table scan.\nJOIN: Shows how tables are joined (e.g., nested loop, hash join), which can impact performance based on the join type and data structure.\nThese elements are organized hierarchically, showing the flow of data and how operations interact. Understanding this helps pinpoint performance bottlenecks.\nAnalyzing Execution Plans for Performance Improvement\nOnce you have generated an execution plan, the next step is analysis. Here are some techniques for identifying performance issues and improving query efficiency:\nIdentify High-Cost Operations\n—High-cost operations are typically the biggest contributors to slow query performance. If you see an operation with a cost much higher than others, it’s likely a bottleneck. Common high-cost operations include:\nFull Table Scans: If the plan shows a TABLE ACCESS FULL on a large table, check if an index could be used to reduce the number of rows scanned.\nExpensive Join Operations: Some joins, like nested loop joins on large tables, can slow down queries. Consider restructuring the query or changing the join type (e.g., hash joins may be more efficient for large datasets).\nEvaluate Join Methods\n—Joins are often at the heart of complex queries, and inefficient joins can drastically impact performance. Here are some common join types in Oracle and when they might be used:\nNested Loop Joins: Typically used when one table is much smaller than the other. This join type works well with indexed tables but may slow down for larger tables.\nHash Joins: Useful for large datasets without indexes. Hash joins create an in-memory hash table, allowing for faster matching.\nIf a join method appears inefficient, consider altering the join type or adding an index to the joined columns.\nCheck for Full Table Scans\n—Full table scans occur when Oracle reads every row in a table. While full scans are sometimes necessary, they can slow down performance, especially on large tables. If a full scan appears avoidable, try:\nAdding Indexes: For columns frequently used in WHERE clauses or joins, indexing can help avoid full scans.\nRewriting Queries: Sometimes restructuring a query or breaking it into smaller parts can reduce the need for a full scan.\nCompare Execution Plans for Query Variants\n—Run the same query with slight modifications, such as different WHERE clauses or added indexes, and compare execution plans. This approach allows you to identify the variations that yield the most efficient plan.\nLook at Estimated Rows vs. Actual Rows\n—The estimated number of rows processed can help predict how well Oracle’s optimizer understands the data. If the estimates seem off (e.g., estimating far more rows than expected), it might be helpful to update statistics on the table or index.\nImproving SQL Performance Using Execution Plans\n—Execution plans can guide various
performance-improvement strategies. Here are a few steps to enhance SQL performance based on insights from execution plans:\nOptimize Index Usage: If a query shows frequent table scans, consider creating indexes on frequently queried columns.\nUpdate Statistics Regularly: Outdated statistics can lead to inaccurate estimates. Regularly update statistics so that the optimizer has an accurate view of table contents, which helps create efficient plans.\nRewrite Complex Queries: If the query is overly complex, try breaking it into simpler steps or using temporary tables. This can reduce the overall load and improve readability.\nUse Bind Variables: Bind variables reduce parsing and can improve performance for frequently run queries by reusing execution plans.\nCheck for Inefficient Nested Loops: Nested loops are sometimes chosen incorrectly by the optimizer. If this appears in the plan and doesn’t make sense given the data volume, consider using hints or adjusting indexes to guide the optimizer towards a more efficient join type.\nIdentifying Root Causes of Performance Issues Using Execution Plans\nExecution plans provide clues for diagnosing performance bottlenecks. Here’s a strategy for identifying root causes:\nExamine Cost and Rows: High-cost operations and high row estimates are primary indicators of inefficiency. Focus on reducing these areas first.\nEvaluate Join Efficiency: Ensure that join methods match the data size. If joins seem to be the issue, experiment with indexes on the join columns or alternative join methods.\nCheck Wait Events: Use Oracle’s SQL Trace and TKPROF tools to analyze SQL execution and identify wait events. This information, along with the execution plan, helps to see where resources are consumed.\nCompare Execution Plan Before and After Changes: When you make an adjustment (e.g., adding an index), generate the execution plan again to confirm that it improves performance.\nConclusion\nUnderstanding execution plans in Oracle SQL is essential for optimizing query performance and ensuring efficient resource use. By carefully analyzing execution plans, database administrators and developers can identify bottlenecks, reduce costs, and keep the database running smoothly.\nRegularly reviewing execution plans not only improves individual query performance but also contributes to the overall health of the database. As you gain experience with execution plans, you’ll find that they offer valuable insights into query optimization, helping you make data-driven decisions for better database management.


Understanding Execution Plans In Oracle SQL...A Guide To Performance Optimization

By Ashwin Kota, HEXstream integrations and analytics lead \nIn today’s utility landscape, ensuring service reliability and effective outage management is essential to meet the demands of regulators and customers. Utility360 is designed to help utilities streamline their data-handling processes (especially for tracking outages) and ensuring compliance with reliability metrics. This post showcases how one utility leveraged Utility360’s ETL capabilities to significantly improve data accuracy, efficiency and customer service.\n1. Improved Reliability with Reduced Data-Processing Times\nThis utility company had previously encountered challenges recalculating outage data, as frequent updates placed a heavy load on their resources. Every recalculation affected the entire dataset, causing delays and limiting metric accuracy. With the Reliability Lock Date feature implementation in Utility360, the utility could isolate recalculations to only the most recent records, reducing processing times and preserving historical data integrity. This change freed up resources to address current outages more efficiently and provided more accurate, real-time metrics without compromising system performance.\n2. Enhanced Data Quality through Automated Audits\nMaintaining data consistency and accuracy can be demanding, especially when manually auditing large datasets. By introducing Audit Dashboard, the utility automated daily checks for discrepancies, such as missing entries or duplicates. This automation greatly reduced the time spent on manual data reviews, enabling the team to focus on priority tasks and ensuring stakeholders received accurate, timely information. Data reliability improved significantly, enhancing confidence in the utility’s reports.\n3. Efficient Outage Management with Switch-Sheet Merging\nDuplicate outage records across various data sources led to inefficiencies and skewed metrics. The Switch-Sheet Merging feature implementation in Utility360 enabled this utility to consolidate duplicate entries, creating a single, unified view of each outage event. This consolidation led to more accurate metrics, particularly for indices like SAIDI, SAIFI and MAIFI, and gave the operations team a clearer understanding of service interruptions. With streamlined records, the team could make faster, data-driven decisions that improved service reliability.\n4. Compliance Improvement with Momentary Outage Exclusion\nTo meet regulatory standards, utilities often need to report only sustained outages. However, including momentary outages (under 60 seconds) previously led to inflated metrics. The Momentary Outage Exclusion feature in Utility360 automatically filtered out these brief outages, ensuring that only relevant data was used in reliability calculations. This enhanced regulatory compliance, provided more accurate reports, and helped maintain a clearer view of genuine service quality.\n5. Real-Time Insights through Customized Dashboards\nOutage management relies on having immediate access to current data. The customizable dashboards in Utility360 allowed this utility’s team to monitor outages, customer impacts, and performance trends in real-time. By tailoring these dashboards, the team could quickly view the most relevant information for rapid decision-making. Over time, the dashboards revealed patterns in outage data, supporting proactive maintenance and improving resource allocation.\n6. Accurate Customer Impact Metrics\nRepeated outages affecting the same customer within short timeframes can distort reliability metrics. The 5-Minute Consolidation Rule logic in Utility360 grouped such outages within a five-minute window as single events, providing a realistic picture of customer impact. This rule improved customer satisfaction by enabling the utility to respond accurately to concerns and present a more truthful measure of service quality, avoiding the misleading effects of inflated metrics.\n7. Incremental Data Loads and Efficient Reporting with Reliability Lock Date\nLarge volumes of outage data can place substantial demands on system resources. With incremental data loading and the Reliability Lock Date feature, Utility360 helped this utility limit recalculations to recent data only, reducing processing load and preserving historical-data integrity. \nThis approach supported consistent, reliable reporting while also saving on computational costs, allowing the utility to analyze long-term trends and inform infrastructure-planning decisions.\nConclusion
\n
By implementing Utility360, this utility transformed its approach to reliability and outage-data management. Key features like automated audits, custom dashboards, and incremental data recalculations enhanced the accuracy of metrics, streamlined operations, and improved compliance. \nOverall, Utility360’s robust ETL capabilities positioned the utility as a proactive, data-driven leader, well-prepared for the challenges of today’s dynamic energy landscape.\n\nCLICK HERE TO CONNECT WITH US TO LEARN MORE ABOUT HOW UTILITY360 CAN OPTIMIZE YOUR OPERATIONS.\n \n


Enhancing Reliability And Outage Management With Utility360’s ETL Tool

By Ashwini Nagendra Prasad, HEXstream lead—Integrations Center of Excellence\nProper IT/OT integration ensures all the systems within an enterprise work in harmony. It boosts productivity and enhances utilities’ day-to-day operations. The goal of integration is to streamline and simplify communication between systems while solving the application-silo problem.\nIntegration is not just about merging technologies or systems; it’s about aligning all aspects of the business—people, processes and culture. A holistic integration approach focuses on the synergy between different departments, ensuring they work together smoothly from day one. By addressing all components of the business, organizations can achieve deeper, more sustainable integration.\nBut if we all agree on the importance of integration, why do so many of us struggle to achieve it?\nI see many companies struggle, since they often simply overlook the importance of integration. Integration can be underestimated as simply data exchange. Companies often fail to implement proper integration strategies. Also, integrating for the sake of technology alone is a missed opportunity; integration should be a means to achieve meaningful business outcomes, not just another box to check.\nWhy Traditional Integration Falls Short?\nTraditional integration strategies have often been linear, focusing primarily on merging systems and aligning processes with limited consideration for other factors like vision, employee engagement and sustainability. While these methods can provide short-term operational efficiency, they often ignore the broader, long-term needs of the organization. Blaming the product for the shortcomings and resistance to changes affects Integration when in reality the integration strategy—not the product—it to blame.\nHow to Approach Integration with a Fresh Perspective?\nTo create a more sustainable and effective integration strategy, organizations must rethink their approach. Here are key elements of integration with a fresh perspective:\nStart with a clear vision: Companies must clearly define what integration will entail at their enterprise, uUnderstanding the business drivers, scope and scale of integration while assessing the current architecture and existing systems.\nInvolve stakeholders early: Engage business teams, operation teams and other stakeholders early on and gather and utilize diverse perspectives.\nLeverage the technology: Designing the correct integration architecture that includes the right type of data exchange/data enrichment/data transformation, the right integration patterns, robust error-handling and exception management, and scalability and performance.\nEmphasize security and compliance to include comprehensive security measures and robust compliance protocols for communcation.\nScalability and future-proofing ensures your approach is change-adaptable and there is room for future extensions and improvements.\nIT/OT integration is complex. That’s just a fact. And while they often focus on technical tasks, companies must not forget the broader, holistic aspects of integration. To maximize the benefits of integration, companies should adopt a comprehensive, strategic approach that considers all facets—technical, business and operational—throughout the entire lifecycle of the project.\nHow is your organization approaching integration? What challenges have you encountered and what strategies have you implemented? Are you struggling with your integration (be it cloud or on-prem or custom built)? I’d love to connect with you to help define the right integration strategy for you and help define the standards and governance needed to standardize your process for the entire project lifecycle. \nClick here to contact me.\n

Rethinking Our Approach To IT OT Integration

By Ashwin Kota, HEXstream integrations and analytics lead \nIn the world of databases, especially with Oracle, there are several features designed to keep data safe and operations smooth. One critical feature is Oracle Archive Logs. Understanding what archive logs are, how they work, and the benefits and challenges they bring can help us see just how valuable they are for data management.\nWhat are Archive Logs?\nArchive logs are essentially copies of Oracle’s redo log files. Redo logs capture every change made to the database, helping Oracle keep track of transactions. When these redo logs fill up, Oracle “switches” to a new redo log file, and the filled log file is then “archived.” Archiving these logs means storing them in a safe location on the disk rather than overwriting them.\nIn simple terms, archive logs act like a historical record of all database transactions. They help create a trail of changes, which is invaluable for data recovery, point-in-time restoration, and much more.\nThe Lifecycle of Redo and Archive Logs\nHere’s a quick look at the stages of redo and archive log management:\nRedo Log Generation: As transactions are processed, changes are recorded in redo logs.\nLog Switching: When the redo log file fills up, Oracle switches to a new one.\nArchiving: The filled redo log file is then archived, creating an archive log.\nLog Cleanup: After successful backups, archive logs can be deleted to free up storage space.\nWhy are Archive Logs Used?\nData Recovery: Archive logs are crucial for recovering lost data. If there’s a system failure or accidental deletion, archive logs can restore the database to a previous state by applying the changes from the archived logs.\nPoint-in-Time Recovery: Archive logs allow for precise recovery to a specific moment. If an error occurs, the database can be restored to just before that time. This is valuable for handling unintended changes or even malicious actions.\nContinuous Backup: Oracle archive logs support backups while the database is live, allowing for continuous operation without downtime. A full backup combined with archived logs means you can restore data to any point after the backup.\nData Replication: Archive logs can replicate data to another database, creating a standby database for disaster recovery or business continuity.\nAdvantages of Archive Logs\nEnhanced Data Security: Archive logs improve data security by enabling a complete history of changes. In case of errors, the database can be restored with minimal data loss.\nFlexible Recovery Options: Archive logs support point-in-time recovery, letting administrators choose the exact state to which the database is restored. For example, if a ransomware attack occurs, archive logs can restore the database to its state before the attack, protecting data integrity.\nReduced Downtime: Because archive logs support live backups, businesses with high availability needs (like banks and e-commerce platforms) can avoid long downtimes.\nImproved Disaster Recovery: By leveraging archive logs, businesses can replicate data to another database at a different location. This ensures that, even in the event of a major disaster, data can still be retrieved.\nDisadvantages of Archive Logs\nHigh Storage Needs: Archive logs take up significant space, which can affect storage costs and capacity. Regular cleanup and storage management are necessary to prevent archive logs from overwhelming server space.\nPerformance Overhead: Managing archive logs requires additional processing power, which can slightly harm overall database performance.\nManagement Complexity: Setting up archive logs can be complex, requiring careful configuration and monitoring. Ensuring proper storage allocation, monitoring, and cleanup processes adds complexity.\nTips for Managing Archive Logs Effectively\nMonitor Storage Usage: Regularly check the space used by archive logs. Use the following command to viewyour archive log destination:\nsql\nCopy code\nSHOW PARAMETER LOG_ARCHIVE_DEST;\nAutomate Backups: Backing up archive logs regularly is essential for both data recovery and space management. You can automate backups using RMAN (Recovery Manager):\nsql\nCopy code\nRMAN> BACKUP ARCHIVELOG ALL;\nAutomate Log Deletion: To prevent archive logs from filling up storage, configure automatic deletion of old logs. The following command deletes archive logs older than seven days:\nsql\nCopy code\nDELETE ARCHIVELOG ALL COMPLETED BEFORE
'SYSDATE-7';\nAlways ensure you have a recent backup before deleting old logs.\nUse a Retention Policy: Set a retention policy for archive logs to keep only the logs needed for the most recent recovery point. This approach helps save space and simplifies log management.\nEnable Archiving Carefully: Enabling archiving requires a few simple steps, but it is essential to do it correctly to avoid errors:\nCheck the Archive Log Mode:\nsql\nCopy code\nARCHIVE LOG LIST;\nShut Down and Mount the Database:\nsql\nCopy code\nSHUTDOWN IMMEDIATE;\nSTARTUP MOUNT;\nEnable Archive Logging:\nsql\nCopy code\nALTER DATABASE ARCHIVELOG;\nOpen the Database:\nsql\nCopy code\nALTER DATABASE OPEN;\nHow to Use Archive Logs for Replication\nIn environments that require high availability, archive logs can be used to replicate the database to a secondary location. This replication helps in setting up a “standby” database, which is a copy of the primary database kept up-to-date by continuously applying changes from archive logs. This setup provides redundancy and allows the secondary database to take over if the primary database fails.\nPractical Scenarios for Archive Logs\nRecovering from Accidental Deletion: If a user accidentally deletes data, archive logs enable recovery to just before the deletion.\nProtection Against Ransomware: If ransomware affects a database, archive logs can be used to roll back to a time before the attack, preserving data integrity.\nDisaster Preparedness: In areas prone to natural disasters, archive logs facilitate quick replication to an off-site database, ensuring data is safe even in emergencies.\nConclusion\nOracle archive logs are a powerful tool in the world of data management. They are essential for ensuring data integrity, supporting flexible recovery options, and maintaining efficient backup processes. With proper configuration and management, archive logs provide enhanced security, reduced downtime, and robust disaster-recovery options, making them invaluable for any critical database environment.\nIn today’s data-driven world, effective use of Oracle archive logs can transform your approach to data management. By enabling archive logging and following best practices for monitoring, backing up, and cleaning up logs, businesses can achieve a highly reliable, secure and resilient database system.\n\nLEARN HOW ORACLE ARCHIVE LOGS CAN BE USED AT YOUR UTILITY. CLICK HERE TO CONNECT WITH US. \n


The Power Of Oracle Archive Logs: Essential For Data Recovery And Beyond

By Satish Saini, HEXstream utilities industry specialist  \nRecent Hurricanes Helene and Milton have highlighted how extreme weather events are affecting America’s power transmission and distribution infrastructure. (In a nutshell, they are exposing serious cracks.)   \nThe efficient management of fleet and field-crew services and resources are critical for quick power restoration, to minimize outages and to ensure safety for citizens. But efficient management isn’t always in place. Smart utilities are leveraging advanced technologies through coordinated and collaborated efforts, enabling them to effectively launch large-scale recovery operations based on improved decision-making insights. \nLet’s explore some of the key technologies being used to manage fleet and field-crew services efficiently: \nGeographic Information Systems (GIS) \nprovide spatial visualization and mapping capabilities, helping identify and prioritize areas impacted by hurricanes with meaningful information on affected grid assets, customer pockets and track-crew locations in real time with updated estimated times of restoration (ETRs). \nMobile workforce & fleet-management systems \nhelp\n \nstreamline field operations by providing dispatchers and field crews with real-time updates, job assignments, and work-order management during emergencies. Equipped with GPS and telematics, these systems include capabilities like mobile workorders, automated job-scheduling, real-time communication, and route-optimization for fleets to access assignments, to report progress, and to update the status of restoration tasks. \nAdvanced Distribution Management Systems (ADMS) including Outage Management (OMS) \nplatforms integrate various utility operations with detailed information on grid assets and infrastructure, including real-time monitoring of the distribution network, fault location and isolation, and some automated switching operations. These systems help utilities direct field crews to the most critical areas, reducing outage duration by detecting which feeders, circuit sections and assets are out of service and suggesting the best course of action to restore power, which guides field crews accordingly. \nAdvanced Metering Infrastructure (AMI)\n data through relevant analytics plays a key role in enhancing outage detection, response and restoration efforts for utilities. By leveraging real-time data from customer smart meters along with their exact locations, utilities can improve operational efficiency, efficiently managing field crews and resources to reduce downtime and enhance customer satisfaction. \nDrone and aerial-inspection technology \nequipped with high-resolution cameras and thermal sensors have the capability to inspect power lines and infrastructure for damage in hard-to-reach or hazardous areas, reducing the need for manual inspections. These tools quicken damage assessment. \nData analytics & data-management systems \nthrough data-storage\n \nplatforms enable utilities to store, access, share and analyze critical data related to network assets, outage history, past events and operational data and crew assignments. Cloud infrastructure ensures data availability and continuity even during power outages, which are key elements in disaster recovery. In particular, cloud-based storage systems equipped with efficient data-synchronization provide a centralized hub for data-sharing and collaboration, ensuring that all crews, whether on-site or remote, are working with consistent and up-to-date information. \nSome of the leading technology solutions and service providers for utilities include ADMS and OMS by various vendors like ABB, GE, Shneider and Survalent. Oracle Field Service Cloud offers end-to-end field-service management with real-time scheduling, routing, and mobile capabilities. IBM Maximo is a comprehensive asset and workforce-management solution that supports field operations and integrates with other enterprise systems.  \nAnd HEXstream’s Utility360 combined with our AMI Analytics offering is an efficient solution to effectively manage field crews, outages ETRs and reliability indices reporting. \nData management and analytics play a pivotal role with the massive volumes of data generated by multiple sources at modern utilities; this data is related to operations, outages, asset-management, field crews and workforce management, and customer-information systems. \nHEXstream, with our decades of expertise in all of these fields, can guide utilities to not just weather the current turbulent state of utilities, but absolutely thrive within it.  \n\nWANT YOUR FIELD CREWS TO WORK SMARTER? CONNECT WITH US HERE TO LEARN HOW HEXSTREAM CAN HELP.\n

Work Is Getting Harder For Fleets And Field Crews, Their Tools And Technologies Are Getting Smarter

By Ashwin Kota, HEXstream integrations and analytics lead \nElectricity is something we often take for granted. When we flip a switch, we expect the lights to turn on and the TV to hum to life. However, maintaining this consistent flow of power is no easy task. Utilities work hard to keep power reliable and avoid interrutions. \n\nCLICK HERE TO GET OUR RELIABILITY-METRICS SPECIAL REPORT.\n\nBut how do they measure how well they're doing? \nIn this article we’ll take a look at four key metrics that utilities use to measure the reliability of electric systems: SAIDI, SAIFI, CAIDI and MAIFI. Let’s see what these metrics mean and why they matter to all of us.\nWhy Track Reliability Metrics?\nReliability in the electric system is all about the dependability of the power supply. Tracking reliability metrics helps utilities figure out where they need to make improvements. The goal is to keep power interruptions—also known as outages—as rare and short as possible. By studying reliability, utilities can see where they are doing well and where they need to make changes. This improves customer satisfaction, as fewer people experience the inconvenience of outages.\nUnderstanding the Key Reliability Metrics\nWhen it comes to reliability, there are four main metrics utilities look at. These metrics, often created using outage data, help utilities understand both the frequency and the duration of interruptions.\nSAIDI (System Average Interruption Duration Index):\n SAIDI measures the average amount of time that customers experience outages in a given year. If the SAIDI score is 120 minutes,that means the average customer experiences a total of two hours of outages each year. Lower SAIDI values mean fewer or shorter outages, which is a good thing.\nSAIFI (System Average Interruption Frequency Index):\n While SAIDI focuses on how long outages last, SAIFI looks at how often they happen. If the SAIFI score is 1.5, it means that, on average, customers experience about 1-2 outages per year. Like SAIDI, a lower SAIFI score is better, as it means fewer interruptions.\nCAIDI (Customer Average Interruption Duration Index):\n CAIDI tells us how quickly the utility can restore power when an outage does happen. To get CAIDI, divide SAIDI by SAIFI; f SAIDI is 120 minutes and SAIFI is 1.5, CAIDI would be 80, meaning the average time to restore power is around 80 minutes. Utilities aim to keep CAIDI as low as possible so customers don’t stay without power for long.\nMAIFI (Momentary Average Interruption Frequency Index):\n Unlike the other three metrics, MAIFI focuses on very brief interruptions (those lasting less than five minutes). These short interruptions are common but can still be annoying. MAIFI helps utilities track these momentary blips in service and figure out ways to reduce them.\nThe Role of Major Event Days (MEDs)\nNot all outages are created equal. Some are caused by routine issues, like equipment failures, while others happen due to large-scale events, like major storms, earthquakes, or wildfires. These large, unpredictable events are called Major Event Days (MEDs). Since these events can cause big disruptions, utilities separate them from the usual daily data to get a clearer picture of regular performance.\nLet’s break down how each metric is calculated with and without MEDs:\nSAIDI\nWith MEDs: When we include MEDs, SAIDI will be higher because these big events cause longer outages. This version shows the impact of rare but high-impact events on the system.\nWithout MEDs: By removing MEDs from SAIDI, we get an average that shows how the system performs on a normal day. This helps utilities understand the typical reliability of their system without the influence of extreme events.\nSAIFI\nWith MEDs: When calculated with MEDs, SAIFI shows how often outages happen, including during extreme events. This helps utilities plan for responses to emergencies.\nWithout MEDs: Removing MEDs from SAIFI shows how often interruptions occur in normal conditions. A lower SAIFI score here is a good sign of the system’s stability day-to-day.\nCAIDI\nWith MEDs: Including MEDs in CAIDI reveals how quickly power is restored after any type of outage, including major events. This gives an idea of how well a utility can respond during emergencies.\nWithout MEDs: Without MEDs, CAIDI shows how efficiently outages are resolved in regular situations. A lower CAIDI here suggests a utility is well-prepared to handle everyday interruptions swiftly.\nMAIFI\nWith MEDs: MEDs have less of an impact on MAIFI, as momentary outages are usually short and not caused by major events.\nWithout MEDs: Since MAIFI focuses on brief interruptions, it remains mostly stable regardless of MEDs. This score primarily helps track minor reliability issues that occur during normal operation.\nThe Difference Between With and Without MEDs is Important\nBy looking at metrics both with and without MEDs, utilities get a fuller picture of their grid’s reliability. Here’s why it matters:\nWith MEDs: Including MEDs in calculations shows the utility’s ability to handle severe disruptions. It helps the utility understand the grid’s resilience and prepare for worst-case scenarios.\nWithout MEDs: Calculations without MEDs provide a baseline, showing how well the system works under normal circumstances. This version of the metrics helps utilities improve routine reliability by targeting areas that need attention.\nTogether, these versions help utilities balance both daily reliability and readiness for rare events, enabling a more reliable and robust grid overall.\nHow Utilities Use Reliability Data\nThe information from these metrics doesn’t just sit on a shelf. Utilities use reliability data to make decisions about where and how to invest in the grid. Here are some examples:\nTargeting Problem Areas: By identifying the circuits with the worst SAIDI and SAIFI scores, utilities can focus their efforts on areas that need the most improvement. These “worst-performing circuits” often become the top priority for repairs and upgrades.\nPlanning for Future Needs: In regions where outages are more common, utilities can strengthen infrastructure or install new technology to prevent future interruptions. This could include adding automated systems that make it easier to detect and respond to outages quickly.\nTracking Improvement Over Time: By comparing metrics over the years, utilities can measure the effectiveness of their investments. A lower SAIDI or SAIFI over time shows that their reliability projects are working, which is a positive sign for both the utility and its customers.\nMoving Toward a Reliable Future\nReliability isn’t a one-time fix; it requires constant attention and improvement. By using these metrics, utilities can understand the strengths and weaknesses of their systems. This knowledge helps them make informed decisions that lead to a better, more dependable power supply. For customers, it means fewer power interruptions and quicker service restoration when outages do happen.\nIn short, SAIDI, SAIFI, CAIDI and MAIFI are the cornerstones of electric-system reliability. By keeping an eye on these metrics, utilities can keep power flowing smoothly and ensure that we rarely have to think about what it takes to keep the lights on.

Navigating The Modern Grid: A Deep Dive Into Utilities’ Reliability Metrics SAIDI, SAIFI, CAIDI & MAIFI

By Siddartha Reddy Katukuri, HEXstream integrations and analytics consultant \nFirst, let’s clarify what, exactly, we mean by “event streaming.”\nIn the utility industry, events refer to any significant data points or changes in state, such as
readings from meters, sensors, pipelines or transactional records. Event streaming enables applications and systems to act on data instantly, allowing
utilities to monitor consumption, predict outages, and optimize resource distribution
in real time.\nBy utilizing event streaming, businesses can unlock the full potential of their data. This
capability allows for real-time monitoring of consumption, predictive
maintenance, and optimized resource distribution, depending on the data at
hand. For example, energy providers can identify grid issues before they
escalate, improving preventive measures and service reliability. In industries
like water management, real-time sensor data can instantly detect leaks,
reducing resource wastage and enhancing service efficiency. \nEvent streaming ultimately enables companies to proactively respond to data-driven insights, fostering better business agility and service quality.\nHarnessing the Power of Oracle Integration Cloud and Apache Kafka for Event Streaming\nOracle Integration Cloud
(OIC) offers seamless integration capabilities with both cloud and on-premises
systems, supported by a wide range of pre-built adapters for various
applications. Apache Kafka’s architecture, optimized for high-scale and
high-performance environments, makes it ideal for organizations managing large
data volumes that require real-time insights.\nOIC’s toolkit extends to process automation, including workflow automation, approval routing, and task management. It simplifies data transformation with visual tools that map data
between systems, supporting conditional transformations. Kafka, in turn, brings
valuable features like event reprocessing and replay from specific points in
time, a critical function for fault-tolerant systems. This capability
complements OIC, which might otherwise need external storage for persistent
event handling.\nTogether, OIC and Kafka
create a powerful solution for modern, data-driven applications by combining
streamlined integration with resilient event streaming. This partnership
enables real-time data processing and efficient handling of large volumes,
enhancing agility and responsiveness across interconnected systems. For
example, in high-demand sectors like oil and gas, the OIC-Kafka combination
facilitates real-time monitoring of machinery and supply chains, enabling quick
responses to operational changes and ensuring continuous production. Businesses
seeking to optimize data flows and build scalable, reactive applications will
find OIC and Kafka to be a highly effective solution.\nEfficient Integration for Business Scenarios with OIC and Kafka\nOIC and Apache Kafka
provide a modern event-processing platform that meets diverse business needs
across industries. With OIC’s prebuilt adapters and Kafka’s effective
partitioning and topic management, this integration provides a flexible,
scalable solution for real-time data routing, workflow approvals, and business
flows. \nFor example, when Kafka
streams its data, OIC can trigger workflows to calculate energy consumption,
bill customers in real time via SMS or email, grid sensors disruptions and—if irregular
data points are detected—OIC workflows can initiate an alert for field teams to
investigate potential issues.\nApache Kafka’s
reprocessing and replay features make it an ideal choice for fault-tolerant
applications, maintaining data continuity during downtimes or recovery.
Although OIC does not inherently store messages, Kafka’s event retention allows
for long-term data storage and replay, adding resilience to event-driven architectures.
This pairing of OIC’s real-time integration strengths with Kafka’s fault
tolerance and data persistence is invaluable for modern applications.\nConnectivity between OIC and Kafka Streaming\nConnectivity between OIC
and Kafka is achieved through OIC’s prebuilt adapters, including the Apache
Kafka Adapter and Confluent connectors. These adapters enable direct
integration with Kafka, facilitating essential operations and enhancing
real-time data streaming and processing capabilities across platforms.\nThis seamless
connectivity is especially advantageous for companies requiring immediate data
availability across platforms such as Azure Event Hub. By leveraging Kafka’s
streaming capabilities alongside OIC’s integration, businesses in sectors like
healthcare can improve patient care through real-time data sharing across
applications. This direct connectivity ensures that systems can communicate
without time delays, helping organizations maintain data accuracy and operational
efficiency. As event data flows seamlessly, business ecosystems become more
responsive to changes, maintaining high performance across multi-cloud or
hybrid environments.\nTo ensure secure,
scalable integration connection, it’s essential to implement encryption,
authentication, and strong monitoring practices. Both OIC and Apache Kafka
provide advanced encryption capabilities including SASL, SSL/TLS, TDE and SFTP
encryptions to protect data throughout its lifecycle, from transfer to storage,
making them suitable for handling sensitive data across various industries,
including finance, utilities and healthcare.\nThe Future of Event Streaming with OIC and Kafka\nFuture advancements in
OIC and Kafka promise even tighter integration and more sophisticated event
handling, setting the stage for innovative applications across industries. \nFor sectors like utilities
and logistics, these advancements will mean greater accuracy in real-time
monitoring of infrastructure, tracking supply chains, and managing production cycles,
ensuring minimal delays and optimized resource allocation. \nAs the event streaming
landscape evolves, businesses that adopt and integrate these tools effectively
will be positioned to lead in their markets, harnessing real-time insights to
drive innovation and maintain a competitive advantage. Embracing the future of
event streaming with OIC and Kafka enables organizations to stay ahead of the
curve, adapting seamlessly to an ever-changing, data-driven world.\n


Integrating Oracle Integration Cloud (OIC) With Apache Kafka For Event Streaming

By Vaishnavi Medikundam, HEXstream BI developer\nOracle has added new facial-intelligence features to Oracle Cloud, offering innovative tools for businesses to enhance their data usage. In the rapidly evolving world of technology, Oracle Cloud is continually innovating, and this new feature is transforming how businesses use data.\nUnderstanding Facial Intelligence\nFacial intelligence goes beyond facial recognition. It's about decoding human expressions, emotions and interactions—providing a deeper understanding of user behavior. Oracle Cloud's new capabilities in this realm promise to change industries that rely on data-driven decision-making. \nKey Features\nOracle Cloud introduces precise facial recognition, emotion analysis, improved user authentication, and analytics/insights. \nOracle Cloud's upgraded facial-recognition system ensures unmatched accuracy, revolutionizing security in sectors like finance and healthcare. The system is equipped to analyze facial expressions, enabling businesses to gauge customer sentiments in real-time. This feature opens new avenues for personalized customer experiences and targeted marketing strategies. \nSay goodbye to traditional passwords. Oracle Cloud's facial intelligence can be integrated and provide a secure and convenient way for users to access sensitive information. \nThe gathered facial-intelligence data isn't just about recognition; it's about insights. Businesses can now leverage analytics to understand customer preferences, optimize workforce management, and make data-driven decisions. \nBenefits Across Industries\n·       Enhanced customer profiling fosters personalized shopping experiences, driving customer satisfaction and loyalty.\n·       Improved patient authentication and emotion analysis elevate healthcare outcomes and patient experiences.\n·      Strengthened security measures (including advanced fraud detection) that can fortify the financial sector against evolving threats. \nOracle Cloud's commitment to innovation continues to empower businesses worldwide. The introduction of advanced facial-intelligence capabilities heralds a new era in data-driven decision-making, promising efficiency, security and unparalleled insights. \nAs we embrace these new capabilities, the future looks brighter and more intelligent than ever before. 

Here's Looking At You: New Facial-Intelligence Capabilities On Oracle Cloud 

By \nGokuleshwar Raju Nannapuraju\n, HEXstream BI/ETL consultant\nAs you may have heard, Oracle has released version 2.9 of its Oracle Utilities Analytics Warehouse (OUA), bringing several new features and improvements. Let’s break down these updates in an easy-to-understand way for non-technical readers…\nSupport for Microservices Architecture for Configuring Oracle GoldenGate\nStarting with version 2.9, OUA now supports the Microservices Architecture for Oracle GoldenGate. This is a modern way to configure and manage Oracle GoldenGate, a tool that helps move data between different databases.\nSo what does this actually mean? \nSimpler and more efficient: Microservices make the configuration and management of data transfers more streamlined and scalable.\nOut with the old: The older "Classic" architecture for setting up Oracle GoldenGate is no longer supported. The configuration module "JKM BI Oracle to Oracle GoldenGate" has been updated to work with this new architecture.\nSupport for Oracle Analytics Server 2024\nOUA 2.9 is now compatible with Oracle Analytics Server version 2024 (version 7.6). It also continues to support the 2023 version. This is important, primarily, because of the enhanced analytics; with this support, users can leverage the latest features and improvements in Oracle Analytics Server 2024, providing better data analysis and visualization capabilities.\nUpdated Supported Platforms\nThe release includes updates on the platforms that are supported, ensuring compatibility with the latest server and client-side technologies.\nServer-side:\nOperating System (Server): Oracle Linux 7.x, 8.x (64-bit), Oracle Solaris 11.4 on
     SPARC (64-bit)\nFMW App Server: WebLogic 12.2.1.4.0\nJDK: JDK 8u411\nOracle Database: Enterprise Edition 19.x, Client for Linux 19\nOracle Analytics Server: 7.0 (2023) and 7.6 (2024)\nOracle Data Integrator (ODI): 12.2.1.4\nApex: 23.2\nOracle GoldenGate Microservices: 19.1\nOracle REST Data Service: 21.4.3\nClient-side:\nClient Operating System: Windows 10 (64-bit)\nBrowser Versions:\nMicrosoft Edge 120.0.2210.77\nMozilla Firefox 121.0\nGoogle Chrome 120.0.6099.109\nSupported Source Application Versions\nThe following source application versions are supported in this Oracle Utilities Analytics Warehouse release:\nCustomer Care and Billing: 2.9.0.1.1\nCustomer to Meter: 2.9.0.1.1\nNetwork Management System: 2.6.0.1.1, 2.3.0.2.0\nWork and Asset Management: 2.4.0.1.1\nMeter Data Management: 2.5.0.1.1\nMobile Workforce Management: 2.3.0.3.0, 2.2.0.x.x\nSupported Upgrade Paths\nThe following upgrade path is available for Oracle Utilities Analytics Warehouse in this release:\nFrom: V2.8.0.1.2 To: V2.9.0.0.0\nSummary\nExisting clients are encouraged to upgrade to OUA 2.9 because the new release supports all the latest versions of source applications and integrates seamlessly with new applications used in OUA. This upgrade will not only resolve existing bugs, but also introduce new features, providing a more robust and efficient analytics solution.\nAs for me, I'm excited to see these upgrades in action across utilities, as they will not only resolve existing issues, but also enhance compatibility with the latest OAS versions and new source applications. This will be incredibly beneficial for end users.\nClick \nhere\n for the official Oracle documentation. \nAnd click \nhere\n to connect with HEXstream to learn how we can help you take advantage of this upgrade. \n
\n
\n


Oracle Updated OUA…So How Do You Benefit From It?

By Arun Kota, HEXstream senior director analytics\nAn intricately crafted disaster-recovery system stands as a vital pillar for utility organizations, ensuring swift and efficient recovery in the aftermath of catastrophes. In the dynamic landscape of IT, disasters take various forms, ranging from network outages and equipment failures to application glitches and natural calamities, which all pose imminent risks to the integrity of your applications. \nAs we can’t predict when a disaster will strike, the best option is to have a plan on how to quickly recover. \nAn important step in disaster-recovery planning is determining the Recovery Time Objective (RTO) and Recovery Point Objective (RPO) for an application. \nRecovery Time Objective (RTO) serves as a crucial metric, reflecting the maximum allowable downtime for a specific application following a disaster, with a general rule that emphasizes shorter RTOs for applications of higher criticality.\nRecovery Point Objective (RPO) signifies the timeframe, post-disaster, during which an application can endure data loss without negatively affecting the enterprise, underscoring the critical need for tailored recovery strategies based on data sensitivity and business-continuity priorities.\nIn crafting a robust disaster-recovery plan, it is imperative to factor in both RTO and RPO, built upon a strategic alignment that harmonizes the goal of timely application recovery with a tolerance for data loss. This dual consideration facilitates the development of a comprehensive, cost-effective, and resilient framework for disaster preparedness.\nMultiple Approaches for Disaster Recovery in OCI\nOracle Cloud Infrastructure (OCI) provides four different approaches in disaster recovery.
Selecting the appropriate disaster-recovery solution for an OCI application hinges on various factors such as availability needs, data durability, as well as the specific requirements for both RTO and RPO, necessitating a thoughtful evaluation to align the chosen solution with the organization’s unique considerations.\n1.    Backup and Restore–This approach backs up critical data and systems into offsite storage on regular intervals. In the event of a disaster, data can be restored from these backups to bring the systems back to their previous states. This is a relatively cheaper approach, but it usually takes more time. The backup-and-restore approach is usually used for noncritical applications.\n2.    Pilot Light–In this approach, a minimal version of the environment is always running in the cloud. In the event of a disaster, the infrastructure can quickly scale up to full capacity. The
RPO and RTO are shorter, but a little costly  compared to the backup-and-restore tactic. The
Pilot Light approach is usually used for critical applications.\n3.    Active/Passive or Warm Standby–In this approach, a fully installed and configured application is on standby and ready to run when required. This setup reduces the time required to switch to a fully operational state in the event of a disaster. The RPO is in seconds and RTO is in minutes; the cost is significantly more expensive than Pilot Light. The
Active/Passive or Warm Standby approach is usually used for critical applications.\n4.    Active/Active–In this approach, the applications are deployed in Active/Active mode and are actively serving. Every transaction is registered  in both applications. Both RPO and RTO are almost zero. This approach has the highest cost, as the applications are running in parallel. The Active/Active approach is used for extremely critical applications.\nOCI disaster recovery can be created using multiple regions. Choosing the right DR approach for an application is very important. The criticality of the applications, data consistency, cost and complexity need to be factored in when deciding the DR approach for an application.\n\nLEARN HOW HEXSTREAM CAN BOOST YOUR EFFICIENCY WITH ORACLE CLOUD INFRASTRUCTURE. CLICK HERE TO CONTACT US. \n\n


Recover Faster! New Approaches To Disaster Recovery With Oracle Cloud Infrastructure (OCI)

By Gokuleshwar Raju Nannapuraju, HEXstream integrations and analytics consultant\nIn this blog post, we navigate through the revolutionary impact of Oracle Cloud Infrastructure–Data Integration (OCI DI), offering a non-technical overview that illuminates its benefits and underscores why it stands as a true game-changer in reshaping the integration landscape for a seamlessly connected future.\nWhat Is OCI DI? \nOCI DI, an abbreviation for Oracle Cloud Infrastructure–Data Integration, serves as a cloud-based ETL (Extract, Transform, Load) tool designed to foster seamless connections between source and target databases. This powerful tool empowers users to efficiently cleanse, transform, and reshape data, streamlining the integration process for enhanced efficiency and reliability.\nUnderstanding Data Integration\nData integration involves the transformation of data from various sources, enabling seamless connectivity between different databases. This process includes connecting to source and target databases, cleansing and transforming the data through mapping and logic definition. Mapping from source to target databases enables data transformation either through manual execution or scheduled jobs based on desired frequencies.\nAdvantages of OCI DI\n1. Zero maintenance: Unlike traditional data-integration tools that require regular patching and service monitoring, OCI DI requires zero maintenance. Users can rest easy, knowing that they are free from any maintenance activities or concerns regarding application versions.\n2. No installation required: OCI DI eliminates the need for time-consuming installations. With a few simple steps, users can create a workspace and promptly start utilizing OCI DI.\n3. Basic configuration: Once the workspace is established, users can effortlessly connect to the source and target databases. OCI DI eliminates the need for complex configurations, ensuring a streamlined integration experience.\n4. Cost-effectiveness: OCI DI is an affordable option for businesses, and its unique pricing model charges users based on data usage and integration needs. There is no need to constantly start or stop services, as costs are calculated according to the actual usage.\n5. Enhanced speed: Regardless of the data volume being integrated, OCI DI utilizes powerful Spark engines in the background, ensuring high-speed execution once the job is initiated. This enables efficient data-integration processes for improved productivity.\nThe Significance Of Zero Maintenance\nLet’s circle back to that zero-maintenance benefit of OCI DI, which alleviates the burdens associated with managing services and ensuring timely startups. Users can enjoy uninterrupted integration capabilities without the need for downtime or patching activities. By eliminating these concerns, OCI DI optimizes business operations and maximizes efficiency.\nIn summary, OCI DI is transforming the integration landscape with its user-friendly features, cost-effectiveness, and maintenance-free experience. By leveraging the power of Oracle Cloud Infrastructure, businesses can enhance their data-integration capabilities and achieve seamless connectivity across various databases. Embrace OCI DI and unlock a new level of integration efficiency for your business.


Transforming Integration: The Evolving Benefits Of Oracle Cloud Infrastructure Data Integration

With the size and scope of collected data expanding with each passing day, it is very important to continually analyze available data to drive better-informed business and policy decisions. To a large extent, handwritten data remains unexplored and unanalyzed. If we can analyze handwritten text data, we can minimize the hurdles and save the manpower involved in digitizing handwritten data.\nIn this blog, we cover how handwritten text data can be processed using Python and Google Cloud Vision. Cloud vision offers pre-trained ML models which are very powerful, and we do not need to do any pre-training.\nBelow is an example of converting simple handwritten text into digital words, which can be easily ingested into a CSV file or database.\nUsing the applications listed in the instructions below, we are able to convert the scanned images or PDF files into digital text. Our approach for this solution requires us to first convert PDF documents to an image format. As we are processing images, it is better to convert all the images to the same size. We then need to define the source from which we are extracting the data from the image. This requires knowing the coordinates of the data to be extracted.\nSteps to Convert Handwritten Text into Digital Data\nThe prerequisites for this exercise are to install Google Cloud Vision, Python 3, Handprint, Keras, NumPy, pandas, pdf2image and cv2.\nConvert the PDF document to an image using the following python command.
\nimages = convert_from_path(file)\nSave the images to a directory so that this will be used for data extraction.
\nfor img in images:
   img.save(path + "\\" + fileName + '.jpg', 'JPEG')\nRead the image using following command.
\nimage = cv2.imread(filename)\n
\nPass the image to the pytesseract to convert the image to data item.
\nTo view the keys, run the below command.\nCheck the extracted text data.
\nExtract the coordinates for the required text using the following command.\nRepeat the same process for all the required text fields.\nUse the extracted coordinates to crop the image for the required field and save the cropped image to a specific location.
\nDownload the credentials file after setting up the Cloud Vision account on Google.
\nProcess these cropped images using Google’s Cloud Vision to extract handwritten values.
\nWe then extract the required data from the Json files.
\nWe have created a Python dictionary to store the extracted information. Convert the dictionary to dataframe for easy processing.
\nAs this example deals with only two images, the instructions did not include many validations. For a real-world application, analysts should validate the extracted data based on confidence levels and filter out the accurate data.\nIn Conclusion\nHandwritten image and text processing is very useful across many industries, including healthcare, insurance, and utilities. Various use cases include invoice processing, processing of inspection forms, employee onboarding, analyzing reviews and survey data and so on. One prescient application for utilities and other organizations relying on distributed field services is the digitization of handwritten notes taken by field technicians when completing repairs or regularly scheduled maintenance. By digitizing these notes, \nand even incorporating process automation to do so\n, utilities are better situated to retain critical handwritten information regarding asset health, maintenance history, and more.

In this blog, we cover how handwritten text data can be processed using Python and Google Cloud Vision. Cloud vision offers pre-trained ML models which are very powerful, and we do not need to do any pre-training.

How to Process Handwritten Text Using Python and Cloud Vision

Selecting the correct machine learning algorithm can be difficult, but doing so is critical in order to answer a given question with high speed and accuracy. In this blog, we will introduce three types of machine learning algorithms and explain how to select the right one when tackling business problems. Then, we share a few real-world use cases for the different algorithm types.\nBefore understanding how to choose the best machine learning algorithms let us briefly look into the types of machine learning algorithms. The machine learning algorithm can be broadly classified into three groups.\nSupervised Learning Algorithm\nUnsupervised Learning Algorithm\nReinforcement Learning Algorithm\nRegression\nK-Mean Clustering\nMonte Carlo\nRandom Forest\nKNN\nMarkov Decision Process\nDecision Tree\nSVD\nQ Learning\nSupervised learning model: Trained with the dataset which consists of labels for both Input and output.\nUnsupervised learning model: Trained with the dataset that has input features but does not have labels for the output.\nReinforcement learning model: Trains itself based on a set of actions and makes the decision.\nNow, let’s learn when to use these algorithms rather than how to use these algorithms. Choosing the correct algorithm is dependent on multiple variables. A good understanding of data science plays a very significant role in categorizing the problem statement.\nThe factors we need to consider while categorizing and solving the problem are:\nKnowledge of Data\n: The data’s structure and complexity help dictate the right algorithm\nAccuracy Requirements\n: Different questions demand different degrees of accuracy, which influences algorithm selection\nProcessing Speed\n: Algorithm choice may depend on the time constraints in place for a given analysis\nVariables\n: The unique features considered while training the model for the optimal result and accuracy help determine the right algorithm.\nParameters\n: Factors such as the number of iterations directly relate to the training time needed when generating output.\nOur first step is to analyze the data and observe the patterns and any hidden insights by visualizing the data. The insights from data visualization will help in making an initial decision on which algorithm to choose for solving the given problem.\nAs an example, if the dataset has input and output labels, a supervised learning algorithm will be best for the problem. Supervised learning can be segregated further based on the type of output. For any numeric output, one should choose a regression algorithm to train the model, but for any class output, one should target a classification algorithm to train the model.\nIf the dataset has no output labels, choosing an unsupervised learning algorithm will be a wise decision, for example, the K-Mean clustering algorithm comes in handy in dividing the dataset into a smaller cluster or set based on similar attributes.\nThe characteristics and behavior of the dataset also play an important role in making the ideal algorithm selection. For example, if the data is sequential, like stock price or heart rate, then the algorithm that fits best is the Markov model and decision tree.\nAnother important factor in deciding the algorithm is speed and accuracy. There is a tradeoff between speed and accuracy. If accuracy is not vital, perhaps when estimating a value, we can reduce the processing time, thus increasing the execution speed. Conversely, we can achieve better accuracy by slowing the execution speed.\nAnother factor that plays an important role in choosing the correct algorithm is the number and types of parameters we pass while training the model. These parameters may include iteration cycle, splitting train and test datasets, error tolerance, and more. The training time for a given model is directly proportional to the number of parameters included. When multiple parameters are required to train the model, SVM (Support Vector Machine) algorithms work well.\nReal-Life Use Cases\nLinear Regression\nThis can be used to predict housing prices based on multiple variables including square footage, the presence of a balcony, the number of bathrooms, total rooms, kitchen area, etc.\nEach of these variables plays an important role in predicting the accurate housing price. A co-relation of 1 indicates a perfect relationship and accurate housing price, anything less than 0.9 can safely be assumed as unreliable prediction.\nLogistic Regression\nCurrently, bank customers face the constant threat of credit card and identity fraud. As a result, the financial sector is working to devise new techniques to avoid these problems. One of the steps banks have taken to reduce fraud is by leveraging logistic regression to build a classifier, known as the LogR classifier, which adapts to the user’s behavior based on his historic data. As a result, financial institutions can more accurately distinguish their true customers from any impersonator.\nSupport Vector Machine (SVM)\nThis helps in handwriting detection using a support vector machine classifier. It has achieved accuracy of up to 96% when \nconverting handwritten notes\n into text-based data.\nRandom Forest\nRandom Forest algorithms can be used to predict equipment failure and schedule just-in-time preventive maintenance. The failure of the equipment, such as utility poles or transformers, can be caused by cracks, corrosion pitting, spalls on rolling surfaces, and more. These failures can be catastrophic and financially devastate a given organization.\nTaking advance of Random Forest machine learning algorithms and AI to monitor these assets may help in \ndetecting failures at their earliest stages\n, reducing or preventing damage to the asset and the organization’s bottom line altogether.\nK-Mean\nUber, Lyft and other rideshare apps utilize K-Mean machine learning algorithms for optimal placing and clustering of their vehicles based on the peak demand. As a result, customers enjoy reduced wait times and helps the rideshare companies dictate surge pricing when supply is low and demand is high.\nIn Conclusion\nAs we have discussed, machine learning algorithms generally fall into one of three categories – supervised learning, unsupervised learning, and reinforcement learning. When addressing different types of business challenges, analysts should carefully consider the characteristics of the data, speed and accuracy requirements, variables, and parameters in order to yield the desired insights.\nBy using the proper algorithms, organizations can expect to benefit from stronger data insights that reflect the unique conditions that guide each business process. As their algorithms continually learn and improve, so will the data-driven decisions that result.

Selecting the correct machine learning algorithm can be difficult, but doing so is critical in order to answer a given question with high speed and accuracy. In this blog, we will introduce three types of machine learning algorithms and explain how to select the right one when tackling business problems. Then, we share a few real-world use cases for the different algorithm types.