Architecture

Category Archives

EAI (Enterprise Application Integration): Enterprise Service Bus (ESB) vs Service Oriented Architecture (SOA)

Let’s evaluate the following question: Is an Enterprise Service Bus (ESB) a technical tool implementation that aids in delivering a Service Oriented Architecture (SOA)?

This article looks at ESB and SOA from a conceptual perspective as opposed to practical implementation. Determining the right solution means for these concepts to be fully investigated and applied to the specific enterprise and its landscape. It is therefore by no means the final say on the subject and does not prescribe a specific tool.

In order to evaluate this statement, one needs to consider what  ESB and SOA respectively are, given a proper definition of the global concept of Enterprise Application Integration (EAI).

Enterprise Application Integration (EAI) – Useful to clearing up the confusion of integration semantics:

eaiLet’s firstly clarify that Sirius does not consider EAI to be an appraoch like SOA or ESB. We consider it to be the broader concept called Enterprise application integration (EAI) since, by definition, EAI is the use of technologies and services across an enterprise to enable the integration of software applications and hardware systems. EAI can therefore be related to middleware as well as distributed technologies. Despite our definition, many vendors offer ‘EAI suites’ that provide cross-platform, cross-language integration solutions.The sharing of data and business processes between applications are the primary purposes for these solutions while also prescribing a set of principles for integration of multiple systems for communication architectures, such as message-oriented middleware  (MOM). In our opinion, this blurs the line between EAI (a concept/framework), which spans various layers of application, business process, data, information and integration architectures, and ESB which is essentially a tool to enablling EAI. Hence, such EAI suites should really be called ESB tools.

Developing EAI technologies involve web service integration, service-oriented architecture, content integration and business process workflow integration. EAI is usually challenged from a technology perspective by different operating systems, database architectures and/or computer languages, as well as other situations where legacy systems are no longer supported by the original manufacturers. Many types of business software such as supply chain management applications, ERP systems, CRM applications for managing customers, business intelligence applications, payroll and human resources systems typically cannot communicate with one another in order to share data or business rules. .

EAI aims to establish the required communication by meeting these challenges through the fulfillment of three purposes, as follows:

  • Data Integration: Ensures consistent information across different systems.
  • Vendor Independence: Business policies or rules regarding specific business applications do not have to be re-implemented when replaced with different brand applications.
  • Common Facade: Developers and users are not required to learn new or different applications because a consistent software application access interface is provided.

The advantages of EAI are clear:

  • Enabling real-time information access
  • Streamlining processes
  • Accessing information more efficiently
  • Transferring data and information across multiple platforms
  • Simplified development and maintenance

Management strategies to enterprise integration must address the four different domains depicted below. Each of these domains represents different challenges.

Relation between the architecture models, viewpoints and integration distribution models from an Enterprise Architecture (EA) perspective.

 

Service Oriented Architecture (SOA):SOA_Metamodel

A service-oriented architecture is essentially a collection of services. These services communicate with each other. The communication can involve either simple data passing or it could involve two or more services coordinating some activity. Some means of connecting services to each other is needed. At a more technical level, SOA is an evolution of distributed computing based on the request/reply design paradigm for synchronous and asynchronous applications which utilise these services to communicate and to share information and functions. SOA can be applicable to a single technology (eg. SAP), with its various modules that communicate via prescibed types of services on fixed standards, or it can be a collection across an enterprise for various applications to integrate via services. SOA developed as an approach to allow scalabiality following the early main-frame computing era where integration challenges resulted into hardly supportable ‘spaghetti’ code, performing non-real-time, flat file processing in order to accomplish integration.

Hence, SOA is one approach to enabling EAI. Clearly, different types of frameworks can be developed to enable EAI. In theory, perfect EAI can only be enabled by tightly controlled frameworks. Frameworks imply governance, which is a big factor of SOA, however,usually lacks completely by design from SOA development. There are different definitions for SOA, however, we persist that no single SOA operates with complete guidance for integration unless clearly stipulated and therefore does not suffice under the definition of a fully fledged framework, catering for all integration that might realistically be required to enable fully fledged EAI. We therefore view SOA as a principle approach to enable EAI requirements. Various different integration technologies and integration methods may still be at play to enable SOA. One may argue that Service orientation is based on the software development concept of Object Orientation (OO). Just like OO can enable re-usability and scalability of code, SOA can enable re-usability and scalability of live, compiled services to remove duplication in information flow across the enterprise. However, just like OO cannot guarantee that code gets re-used, similarly SOA cannot guarantee that services get re-used, since SOA’s set of services could potentially all still just be enabling point-to-point integration as opposed to agile integration (plug-and-play). This is where an Enterprise Service Bus (ESB) might come in handy to either compliment or take over from the distributed, standardised services composing your SOA model.

 

Enterprise Service Bus (ESB):

gartner-ipaas-2016An Enterprise Service Bus (ESB) is fundamentally an architecture. However, numerous tools have now been developed to practically enable it as a freestanding technology to solve EAI challenges. It is a set of rules and principles for integrating numerous applications together over a bus-like infrastructure which can also house the various services contained in SOA, together with other integration mechanisms in order to centralise and standardise integration across the organisation. Essentially, ESB provides Application Programming Interfaces (APIs) for developers to create services and send messages between services with real-time integration. Various ESB products enable users to build this type of architecture, but vary in the way that they do it and the capabilities that they offer. The core concept of the ESB architecture is that you integrate different applications by putting a communication bus between them and then enable each application to talk to the bus. This decouples systems from each other, allowing them to communicate without dependency on or knowledge of other systems on the bus. The concept of ESB was born out of the need to move away from point-to-point integration as well as improvements to SOA, which can become brittle and hard to manage over time. Point-to-point integration results in custom integration code being spread among applications with no central way to monitor or troubleshoot. This is often referred to as “spaghetti code” and does not scale because it creates tight dependencies between applications. ESB solves this by forcing centralised control for integration testing. Essentially, ESB should be intended to form the backbone of any SOA architecture to enable centralised and standardised control, in turn meeting the challenges and gaining the advantages of properly governed EAI. ESB properly spans the various domains of integration as opposed to only grouping services. We need to place a cautionary note here that no ESB will cater for every aspect of integration across all technologies and at maximum offers control for the majority of integration points within the ringfenced control of an organisation. Some forms of batch processing might still be required internally due to complexity, skill or speed constraints. Let’s discuss the overlap of ESB and ETL for big data briefly in order to eliminate confusion on what this now means in the cloud- and big data context.

esb[1]

 

ESB and SOA in the new Big Data Context: APIs and ESBs  can be used to Simplify Data Integration for ETL in the cloud computing domain

The increase in popularity of Application Programming Interfaces (APIs) has made it much easier to create connectivity. With APIs, developers can access endpoints and build connections without having in-depth knowledge of the system itself, simplifying processes tremendously. As Extract-Transform-Load (ETL) data tools remained focused more towards BI and big data solutions, and as traditional operational data integration methods become outdated with the rise in popularity of cloud computing, ESBs become better options to solving connectivity requirements.

ETLAn enterprise service bus (ESB) provides API-based connectivity with real-time integration. Unlike traditional ETL tools used for data integration, an ESB isolates applications and databases from one another by providing a middle service layer. This abstraction layer reduces dependencies by decoupling systems and provides flexibility. Developers can utilize pre-built connectors to easily create integrations without extensive knowledge of specific application and database internals, and can very quickly makes changes without fear of the entire integrated system falling apart. Shielded by APIs, applications and databases can be modified and upgraded without unexpected consequences. In comparison to utilizing ETL tools for operational integration, an ESB provides a much more logical and well defined approach to take on such an initiative. On the down-side, batch data transfers via ESB can become painful and ETL developers such as Business Intelligence (BI) developers, who are used to typically accessing large quantities of data directly, may be prone to circumvent the ESB, therefore forcing integration architecture back to only being conceptual and responsive rather than physical and in control. It therefore remains imperative for collaboration between BI and integration team leads, since the most large scale ESB tools now include options to connect to relational databases as well as emerging Big Data platforms. For instance, take a look at the following comparison between Biztalk and Informatica.

In Summary

  • SOA is an architectural approach where we expose ‘service’ in a coarse-grained manner whereas ESB is a technical implementation that aids in delivering a SOA.
  • SOA brings cost effective, reusable and low lead time solutions to an organization whereas ESB enables low cost integration and benefits companies with limited IT resources.
  • SOA is a way of building the next generation of applications from ‘lego blocks’ called services whereas ESB is a piece of infrastructure software that provides APIs for developers to create services and send messages between services.
  • SOA is just like a car and ESB is like a road on which this car runs.
  • SOA is an architectural model for implementing loosely coupled service based applications whereas ESB is a piece of infrastructure software that helps developers to develop services, and communicate between services through suitable APIs.
  • Certain ESB tools are now allowing SOA to be implemented while also enabling big data ETL via the necessary exposed layers for data enquiry

 

A senior advisor will contact you as soon as possible. Contact: advisory@siriussa.comRequest Assistance Here!


Data Architecture VS Information Architecture

dataconnectionsPlease note that our philosophy on the subject enables the TOGAFF 9 Capability Based Planning approach. This enables the CIO / CDO  with proper governance from an enterprise architecture and IT perspective since capability-based planning is a powerful mechanism to ensure that the strategic business plan drives the enterprise from a top-down approach. It is also adaptable with capability engineering to leverage emerging bottom-up innovations. This aligns with the Sirius approach of planning, engineering and delivery of solutions to enable strategic business capabilities to the enterprise.

Data vs Information


binary-blue-data-infoWe are often tasked with this question – what is the difference between data and information architecture? This seemingly simple question (from a technical perspective) might be met with complicated, conflicting answers. Let’s simplify this into something that aligns well with both common sense and industry norm.

Most professionals who started their careers in the data field will agree that two decades ago this was a much simpler question. Not only has the lines between terms such as data and information been blurred due to differing business cases and applications, but the value association of ‘data’ and ‘information’ may differ between businesses. This makes the question relevant within the current ‘big-data’ paradigm.

Let’s first, therefore, clarify our Sirius definition of the difference between data and information. This definition aligns well to all the major, generally accepted frameworks on the subject.:

Information is data put into action. Where a specific data element may exist in any form of format or another, it has lesser business value until it is integrated with other data elements into a package of data elements. This package can be described as an information package. Hence, information is data with context. Knowledge and insights are further derivations of information, yet, in the IT context, it is still based on data“. 

The two forms of ’01011000…’-Architectures

data-storage If ‘data’ refers to singular items of values without context, then Data Architecture is necessary to contain and organize the various data resources into a manageable data environment.

Likewise, if ‘information’ refers to contextualized data, then Information Architecture is necessary to combine those resources into a structure that allows the dissemination of that information to be created, shared, analyzed, utilized, and governed throughout an enterprise, across all lines of business, within all departments, with confidence and reliability.

Practical application (Data Architecture VS Information Architecture)

information-worldThe Data Architect focuses on the technical depth implied by and associated with the design, physical structuring, storage, movement, planning, backup, retention, dissemination and classification of all data elements within the enterprise and may or may not be required to engage across business functions. Such functions would typically result into technical specifications during business application design.

The Information Architect focuses on the technical, business and governance matters implied by and associated with the design, overlap, contextualization, usage, security, sharing, discontinuation, valuation and publication of all information sets across the enterprise and across business functions. The resulting architecture should form part of the greater data governance strategic pack as well as the enterprise architecture and enable the data architects with clearly contextualized application required from resulting information sets.

These two roles have clearly segregated areas of control as well as points of engagement for overlap. If segregated, it is not recommended for one role to report to the other, but it is highly recommended that both report to the ‘Chief Data Officer’ , ‘Information Management Executive’ or ‘Enterprise Architect’.

Need assistance? Get in touch!

A senior advisor will contact you as soon as possible. Alternative: 0861 222 982 / advisory@siriussa.comRequest Assistance Here!


Business Intelligence Technology Selection – 5 Crucial points for Executives

Sirius-Flex-for-BI_smallWe have published previous articles on the current reality of BI project challenges in the African context. Let’s do a quick recap on our 2015 findings to date on Business Intelligence Technology Selection – 5 Crucial points for IT and business Executives are listed. The aim of this article is to serve the project teams at our clients with some key learning points for starting off with BI technology selection. The goal is to promote time savings for BI selection and implementation and to speed up the process to provide the organization’s access to trusted, balanced and centralized information. This article does not advise our audience on any specific technologies or how to go about the selection itself.

 

1.    In mind – project management challenges: Scope, Cost and Time. These three, hand-in-hand, are still the biggest competitors to quick BI value realization. Project Resourcing is now less of a problem for the generally implemented technologies in the African space. The technology you select, within the context of your organisational domain and architectural landscape, will largely determine the significance of these challenges during implementation. The reasons for these three elements can be further expanded upon in the next four.

customermatrix

2.    Design as consideration during selection: Architecture and Integration challenges are the main contributors to the extension of scope and time in major BI implementations. A key learning to take into your project from the failures of others, regardless the technology for your core BI solution, is to ensure that the exact amount of integration required in your application landscape as well as how that couples with current architectural standards such as centralized master data management(MDM) and other components in a specific architectural model (eg. Service Oriented Architecture (SOA)) is scoped out well enough even before the selection and blueprinting is done.  Having a clearly defined view of your integration scope and effort before entering the selection and design phases leads to less changes and scope creep during implementation.

 

3.    Broader architectural change: The landscape for BI implementations have changed from an architectural perspective. The average large corporate now spends 30% of their annual IT budget on landscape redesign, optimization and revitalization to keep in line with sudden strategic business changes and unforeseen economic pressures leading to corporate restructuring, relocation, mergers and acquisitions etc. The consequence for BI is that it tends to play catch-up now more than before. There is no simple solution be it through governance or standard management practices to ensure this causes less delays on BI projects. The answer resides in having a truly scalable, yet loosely coupled BI design framework right at the core where the integration between data components become your reality. We strongly recommend that you do not engage any specific technology until your project team understands your architectural principles and challenges. Ensure that your design is ready for rapid change. The entire structure from base data through staging to cubes now, more than ever before, requires a flexible, ‘plugin-plugout’ design. More than a stock standard Kimble or Inman design is required to ensure that flexibility and scalability becomes an enabler to BI for sudden major landscape changes.

christmasbells

4.    The belief that BI can ‘quickly become “real-time” and “in-memory”’. These words have become like the silver bullets, yet ever still not piercing the layers of IT kevlar compounding its challenge. Yes, there are numerous solutions to enforce the possibility of this belief. However, big-data still bares its constraints, not only by Moore’s law, but also by solid architectural common sense. Where data volume meets constraints which are uncontrolled by your chosen BI technology (eg. Large batch loads, flat file processing, delayed message queues, network lag and infrastructure issues, legacy integration), real-time is always an actual idea but still a very idealistic concept. The African context does not yet completely lend itself to real-time BI as much as the major BI technologies at hand may enable real-time and in-memory analytical capabilities by themselves. Senior Executives need to consider the constraints in their IT landscape before choosing to invest the ‘major bucks’ on the latest, most niche BI technology promising to deliver cross-organisational real-time performance views and dashboards.

 

5.    Executive support: Mainly for reasons 2 and 4 above, executives are often frustrated with the amount of time and money invested to deliver singular, static reporting which could effectively be delivered with much more rapid and less costly approaches. Focus on providing your stakeholders with a proper short- to medium term VS long term BI strategy before engaging on a suitable technology investment, regardless the ‘bells and whistles’ on offer.

 

Need assistance? Get in touch!

A senior advisor will contact you as soon as possible. Alternative: 0861 222 982 / advisory@siriussa.comRequest Assistance Here!


Enterprise Architecture VS Solutions Architecture – an overview

Sirius-Archteam_smallIf you are looking for a comprehensive meta-model describing detailed causation relationships and semantic relationships between activities and artifacts in Enterprise Architecure, don’t look here. Instead, we suggest you start with an established model such as Nick Malick’s Enterprise Business Motivation Model (EBMM) or the Zachmann Framework. The purpose of this article is simple – it provides a pragmatic description of key architectural components as well as the difference between the roles of the Enterprise and Solutions Architect. Let’s explore the elements involved first.

Key Role Elements Involved

Business Architecture is concerned with the alignment of business and its dynamic capabilities (principally in most organisations specifically the alignment of business and IT). It achieves this by forming a number of holistic views that enable scenario planning, impact assessment and decision making to be achieved. These views include the business processes, the organisation structure, the application portfolio and the capabilities view of the organisation.

Maintained in step with the business architecture, the Information Architecture addresses the entities of the business, their structure, meaning and canonical forms. Information Architecture also considers the disposition and placement of data, its rules and structure and constraints placed on it. It is concerned with the elaboration of a ubiquitous language that can be shared between all stakeholders, including business and architecture.

The Technology Architecture steps from the domain of What to How. It is concerned with elaboration of the organisation strategy into a set of reference architectures, patterns and guidelines. These are supported by development standards, the service catalogue underpinning capability reuse and a base architecture model that provides a conceptual reference view for the organisation’s architecture fulfilment.

Essential to realizing the target state of any solution, Data Architecture describes how data is processed, stored, and utilized in an information system or across systems. It provides criteria for data processing operations so as to make it possible to design data flows and also control the flow of data in the system.The Data Architect is typically responsible for defining the target state, aligning during development and then following up to ensure enhancements are done in the spirit of the original blueprint

The functions defined

All of these artifacts frame and facilitate the work of the Solutions Architect. The interaction between disciplines is supported by Vitality and Governance processes that not only ensure conformance, but also continued development of the Architecture.Hence, the role of the Solutions Architect is more focussed and suited to project-related than organisational governance initiatives.

Tsirius business solutions icon architecturehere are many different interpretations for the role of an Enterprise Architect. The role between enterprise and solutions architcture is even often blurred. This is understandible, given “enterprise architecture” have many different meanings despite well-known frameworks like Zachmann and TOGAF. We often find that there are only particular aspects examined to properly define these roles. To compound the issue, information technology professionals scope enterprise architecture differently. Some IT professionals view enterprise architecture as a more structured activity with many moving parts such as architecture strategy, architecture patterns, principles, architecture design, business architecture, and IT governance. Others may just focus on the technical aspects rather than the business, operational, or strategic aspects. Many view enterprise architecture as an IT-only issue. In many cases, it is important that the enterprise architecture group speak in both business and IT terms. In our opinion, most importantly, EA should be neutral in their functions in order to ensure architectural continuity and proper usage of its functions as opposed to being a ‘red tape’ function which can be seen by IT and business stakeholders as a ‘reactive obstacle’ to decisionmaking. The function of EA should thus be well balanced between governance and practical guidance to ultimately ensure that through governance  the implementation of its selected/defined framework, throughout design to implementation, the solutions architects are enabled and guided. Yet its challenge is to remain relevant and actionable as opposed to its functions purely being focussed on technical aspects, playing ‘catch-up’ and documenting architectural outcomes in architecture models.

Bottom line: The difference summarised

In simple terms, the difference can hence be denoted as a function of project implementation through framework facilitation for Solutions Architecture, while the dominant theme of Enterprise Architecture remains focused toward IT and business governance though defining the same framework in line with IT and business strategy. The two should in fact often overlap for the sake of consistent alignment and implementation of architectural frameworks, however, the level of its audience and stakeholder engagement should be split between top management, senior management and project-related stakeholders.

Need assistance? Get in touch!

A senior advisor will contact you as soon as possible. Alternative: 0861 222 982 / advisory@siriussa.comRequest Assistance Here!