Monday, 21 April 2014

Integrating Tivoli Directory Server (TDS) with IBM Cognos BI to provide secure & multitenant environment


IBM Cognos Business Intelligence (BI) is a enterprise class, web-based, integrated business intelligence suite by IBM which provides toolset not only traditional BI capabilities like reporting, analysis, scorecarding, monitoring of events and metrics but also expands these capabilities with planning, scenario modeling, real-time monitoring, and predictive analytics. These capabilities deliver an easy-to-use and unified experience that is collaboration and social networking enabled. The IBM Cognos BI has Service-oriented architecture - designed for scalability, availability, and openness.

IBM Tivoli Directory Server (TDS) is a powerful and authoritative enterprise directory infrastructure that is a critical enabler for enterprise security. It is an important part of the IBM Security Integrated Identity Management portfolio. It plays a key role in building the enterprise identity data infrastructure for applications such as identity management, portals, and web services. It provides a server that stores directory information using a DB2 database. It also provides a proxy server for routing LDAP operations to directory servers with database. IBM Security Directory Server provides client utilities and graphical user interfaces (GUI), such as Instance Administration Tool (idsxinst) and Configuration Tool (idsxcfg), to manage servers.

IBM Tivoli Directory Server provides:

  • Industry-standard architecture and broad platform support for a range of operating systems and applications and a variety of heterogeneous environments.
  • Strong scalability and flexibility to support hundreds of millions of entries using IBM DB2 technology and a built-in proxy-server.
  • Availability to support an identity data infrastructure for global online applications such as consumer-driven web services.
  • The ability to help you manage identities in the cloud.
  • Robust auditing and reporting that provides insight with connectivity to IBM QRadar SIEM and greater visibility into repository with sample reports.

You can use IBM TDS to provide a trusted identity data infrastructure for authentication. As we know Cognos BI doesn’t provide its own authentication mechanism but leverage your existing mechanism which you are using across enterprise applications. In this blog article our objective is to leverage existing security features for authentication and data transfer of TDS based LDAP with IBM Cognos BI to order to secure BI assets and setup multi-tenancy environment.

This blog article describes the step by step procedure for –

1)     Setting up TDS 6.2 environment on Windows 7 OS

2)     Integrating IBM Cognos BI 10.2.1 Server with TDS 6.2.

3)     Enable Multitenancy for Cognos BI environment

Also see –





Setting up TDS 6.2 Environment on Windows 7 OS

1)     Installation steps are pretty easy and intuitive for TDS 6.2 by just double clicking install_tds.exe file but if you are using later editions then you need to install it thru IBM Installation Manager. Steps can be found here - http://publib.boulder.ibm.com/infocenter/tivihelp/v2r1/topic/com.ibm.IBMDS.doc_6.3.1/concept/c_ig_InstallationWithIBMInstallationManager.html

2)     On the completion of installation, you can see ‘IBM Tivoli Directory …’ windows services (Start->Programs->Administrative Tools->Services). The default port used by TDS for LDAP service is 389.




3)     To create and manage directory instances click on “Instance Administration Tool” from “IBM Tivoli Directory Server 6.2” folder in Start Menu - > All Programs as shown in snapshot.




4)     Click on “Manage…” button. It’ll open TDS Configuration Tool. Besides getting info about your setup you can also perform many tasks listed on left side panel as shown in below snapshot. Click of “Manage suffixes” task.




5)     We need to add “dc=example,dc=com” as a new suffix before importing our example LDIF. After successful addition you would see it in “Current suffix DNs” list.


6)     Below given is the glimpse of sample LDIF, you can download the attachment (http://www.megafileupload.com/en/file/521432/IBM-TDS62-ldif.html) and change is as per your requirements. I’ve created 11 users having userid admin, user1 – user10 with password – “password”. Lets click on “Import LDIF data”.


7)     Import sample LDIF file.



8)     On successful restoration start the server instance from “Manage Server State” task on the left side, shown in below snapshot.




Integrating IBM Cognos 10.2.1  BI Server with TDS 6.2

It is assumed that Cognos 10.2 BI server is already installed and is in working condition. Open ‘IBM Cognos Configuration’ from Start -> All Programs -> IBM Cognos 10 – 64.

1)      In the Explorer window, under Security, right-click Authentication, and then click New resource -> Namespace.

In the Name box, type a name for your authentication namespace (we used ‘IBM_TDS62’ here) and in the Type list, select ‘LDAP – Default values for IBM Tivoli’ and click OK.




2)      Select the newly created namespace. In the ‘Resource Properties’ window in right, for the Namespace ID property, specify a unique identifier for the namespace as TivoliLDAP is assigned in the below screenshot. All entries with Red arrows are manually provided to integrate with the TDS environment we created in above section.




 3)     If you want the TDS to bind to the directory server using a specific Bind user DN (Distinguished Name) and password when performing searches, then specify these values.



If no values are specified, the LDAP authentication provider binds as anonymous.

If external identity mapping is enabled, Bind user DN and password are used for all LDAP access. If external identity mapping is not enabled, Bind user DN and password are used only when a search filter is specified for the User lookup property.

4)     You can use user attributes from TDS in namespace configuration. To configure this, you must map these attributes with appropriate property name as shown in below snapshot. ‘Custom properties’ would be available as session parameters through Framework Manager.

 

 5)     From the File menu, click Save. Test connectivity to the namespace by right clicking on the name under Security, Authentication and selecting test. If the test is successful, this message box will appear.



If you want to disable anonymous access, make sure you disable it by setting ‘Allow anonymous access?’ property for ‘Cognos’ namespace as shown below in snapshot. 



6)     Restart Cognos service from toolbar. 


7)     Now anyone who wants to access Cognos (http://localhost/ibmcognos), would be asked for authentication credential. Let us login with LDAP administrator credential.



Directory administrators would have Cognos admin privileges. Go to Cognos administration.


8)     In ‘IBM Cognos Administration’, explore ‘Users, Groups, and Roles’ under ‘Security’ tab. One can see the new namespace (IBM_TDS62). Click on it to view all users belongs to the directory.


Administrator now can assign different privileges and roles to these directory users as per application security requirements by setting relevant properties. Once security permissions are assigned, LDAP users are ready to use Cognos BI. For more information on security, please refer to “IBM Cognos BI Administration and Security Guide”.

Enable Multitenancy for Cognos BI environment

1) We need to set multitenant properties from IBM Cognos Configuration tool to enable this feature.  In IBM Cognos Configuration tool, select Security->Authentication->IBM_TDS62 in Explorer (left pane) window. Now select ‘Advanced Properties’ from right window (Resource properties) and add two new values before pressing OK button -

a)     Name – ‘multitenancy.TenantPattern’ value – ‘~/parameters/tenantID’

b)     Name – ‘AdditionalUserPropertiesToQuery’ value – ‘parameters’



2) Now, select ‘Custom Properties’ from right window (Resource properties) and add a new value –

Name – ‘tenantID’ value – ‘l’




3) From the File menu, click Save. Test connectivity to the namespace by right clicking on the name under Security, Authentication and selecting test. If the test is successful, this message box will appear.


4) Save the configuration and restart Cognos service. Your Cognos multitenancy feature is enabled. 

There are many tasks follows this step to realize benefits of multitenancy in BI project. Please refer to my previous blog article http://vmanoria.blogspot.in/2014/03/ibm-cognos-bi-setting-up-multi-tenancy.html to see how to manage/administrate multi-tenant environment.

Wednesday, 19 March 2014

Stream Computing for Real-time Analytics: Overview, Integration and Use Cases



 
Stream computing delivers real-time analytic processing on constantly changing data in motion. It enables descriptive and predictive analytics to support real time decisions. Stream computing allows you to capture and analyze all data - all the time, just in time. Relational databases and warehouses find information stored on disk. Streams analyses data before you store it. Key points here are -

 

1) Stream is the right capability when the primary big data challenge is analyze data that is in motion (Velocity) – because the business imperative requires a real-time response/action based on analyzing the data or the data is very large and want to more cost-effectively filter and remove data before moving into your data warehouse or Hadoop system.  It can handle continuous or bursty streams of data – millions of events per second with microsecond latency.
2) Streams can process any type of data (Variety) – audio, video, network logs, sensors, social media such as Twitter, in addition to structured data.
3) And, Streams is designed to scale to process any size of data from Terabytes to Zetabytes per day

Stream computing changes where, when and how much data you can analyze. Store less, analyze more, and make better decisions, faster with stream computing. The benefits of streaming analytics are immediately obvious.  Dramatic cost savings by analyzing data and only storing what is necessary.  The ability to detect and make real-time decisions, results in customer retention to detect fraud to cross-selling a product. 

IBM InfoSphere Streams for Stream Computing

IBM InfoSphere Streams is an advanced analytic platform that allows user-developed applications to quickly ingest, analyze and correlate information as it arrives from real-time sources. InfoSphere Streams is designed to handle very high data throughput rates, up to millions of events per second. A market leader in providing sophisticated analytics for IoT, IBM received the 2013 Ventana Research award for Operational Intelligence in the IT Innovation category for InfoSphere Streams.

Core highlights are -

  • Perform advanced real-time analytics on data in motion
  • Rapidly ingest, correlate and continuously analyze a massive volume and variety of structured and unstructured streaming data as it arrives from thousands of sources
  • Make real-time predictions and discoveries as data arrives
  • Visualize data easily with drag-and-drop development tools
  • Detect and respond to critical events immediately
  • Learn and update models for future analysis and trend prediction with cognitive computing

InfoSphere Streams helps you:
  • Analyze data in motion—provides sub-millisecond response times, allowing you to view information and events as they unfold. Tools facilitate sophisticated analytics, such as geospatial, voice, image and text, and also update models on the fly.
  • Simplify development of streaming applications—uses an Eclipse-based integrated development environment (IDE). Developers are able to easily and rapidly build applications and connect to new data sources. Drag-and-drop editors, wizards, visualization tools, and runtime monitoring and debuggers are available.
  • Extend the value of existing systems—integrates with your applications, and supports both structured and unstructured data sources. The supporting infrastructure adapt to rapidly changing data formats, types and messaging protocols. It also read from and writes to a vast number of data sources. A massively parallel architecture is designed to deliver unlimited compute potential.
 
IBM Infosphere Streams capabilities are designed to work together and with existing bigdata & analytics applications such as BI and predictive analytics.  Here’s an example scenario:

1)     Historic data is stored in the DB/warehouse (DB2, Infosphere Warehouse, Informix, Oracle, solidDb, MySQL, SQLServer, Netezza etc.) where interesting patterns are detected using database toolkit operators, such as the pattern of credit card transactions that would indicate possible fraud. Support for XML allows developers to fuse a broader range of traditional and untraditional data.

2)     IBM SPSS leverages IBM SPSS Modeler to develop and build predictive models, and then deploy them using the SPSS Scoring Operator. The PMML models are then imported into InfoSphere Streams Studio to generate Streams programs that are executed to score the incoming records in real time without suspending InfoSphere Streams applications.

3)     Additional data sources such as RFID tags, blogs, or other information might be used to improve the confidence levels of the scoring algorithms.

4)     These measures can be sent to Dashboards like IBM Cognos Real Time Monitoring or business process management (BPM) systems to trigger business processes to take immediate action as required.

5)     IBM InfoSphere BigInsights lets you store streaming data in an enterprise-class Hadoop environment for additional analysis or historic retention. InfoSphere Streams and InfoSphere BigInsights use the same advanced text analytics capabilities to simplify natural language processing applications for both data in motion and data at rest. In addition, InfoSphere BigInsights can be used to augment streaming sources with contextual information, and users can visualize InfoSphere Streams data in the InfoSphere BigInsights console.

6)     Streams real-time analytics can be integrated with ETL solutions like IBM DataStage helps get more timely results and offload some analytics load from the warehouse. IBM InfoSphere DataStage helps users perform deep analysis and gain additional insight using contextual and source data from other parts of the infrastructure.

7)     Messaging queues allow InfoSphere Streams to receive data from or send data to IBM WebSphere MQ, IBM MessageSight and Java Messaging System (JMS) offerings.

8)  IBM InfoSphere Data Explorer enables users to visualize InfoSphere Streams data in the InfoSphere Data Explorer CXO dashboard and add streaming data to the InfoSphere Data Explorer index. 
 
Stream computing use cases
When companies can analyze ALL of their available data, rather than a subset, they gain a powerful advantage over their competition. Many customers are seeing tangible ROI using IBM Stream solutions to address their big data challenges:
  • Healthcare: 20% decrease in patient mortality by analyzing streaming patient data
  • Telco: 92% decrease in processing time by analyzing networking and call data
  • Utilities: 99% improved accuracy in placing power generation resources by analyzing 2.8 petabytes of untapped data
Below are few cross-industry scenarios best suitable for stream computing –

1)     Know Everything about your Customers
·         Social media customer sentiment analysis
·         Promotion optimization
·         Segmentation
·         Customer profitability
·         Click-stream analysis
·         CDR processing
·         Multi-channel interaction analysis
·         Loyalty program analytics
·         Churn prediction

2)     Innovate New Products at Speed and Scale
·         Social Media - Product/brand Sentiment analysis
·         Brand strategy
·         Market analysis
·         RFID tracking & analysis
·         Transaction analysis to create insight-based product/service offerings

3)     Instant Awareness of Risk and Fraud - Lower risk, detect fraud and monitor cyber security in real time. Augment and enhance cyber security and intelligence analysis platforms with big data technologies to process and analyze new types (e.g. social media, emails, sensors) and sources of under-leveraged data to significantly improve intelligence, security and law enforcement insight.
·         Multimodal surveillance
·         Cyber security
·         Fraud modeling & detection
·         Risk modeling & management
·         Regulatory reporting

4)     Exploit Instrumented Assets
·         Network analytics
·         Asset management and predictive issue resolution
·         Website analytics
·         IT log analysis

5)     Run Zero Latency Operations
·         Smart Grid/meter management
·         Distribution load forecasting
·         Sales reporting
·         Inventory & merchandising optimization
·         Options trading
·         ICU patient monitoring
·         Disease surveillance
·         Transportation network optimization
·         Store performance
·         Environmental analysis
·         Experimental research

Here’s few usecases in industries to get an idea about the breadth of possibilities that stream technology along with other bigdata products can offer. To explore more details, click on the industry title below.
 

Automotive

  • Data warehouse optimization
  • Predictive asset optimization
  • Connected vehicle
  • Actionable customer insight

Banking

  • Optimize offers and cross sell
  • Contact center efficiency and problem resolution
  • Payment fraud detection and investigation
  • Counterparty credit risk management

Consumer Products

  • Optimized promotions effectiveness
  • Micro-market campaign management
  • Real-time demand forecast

Energy and Utilities

  • Distribution load forecasting and scheduling
  • Create targeted customer offerings
  • Condition-based maintenance
  • Enable customer energy management
  • Smart meter analytics

Government

Geospatial analysis requires complex mathematics such as set theory and geospatial geometry. It is used for location intelligence and location-based services for security and surveillance, geographic information systems, traffic patterns and more. The city of Dublin, Ireland, uses InfoSphere Streams to analyze 50 bus locations per second for its fleet of roughly 1,000 buses. Check it out - http://www-01.ibm.com/software/success/cssdb.nsf/CS/RNAE-9C9PN5?OpenDocument&Site=software&cty=en_us
  • Threat prediction and prevention
  • Social program fraud, waste and errors
  • Tax compliance - fraud and abuse
  • Crime prediction and prevention

Healthcare

  • Measure and act on population health
  • Engage consumers in their healthcare
  • Health monitoring and intervention

Insurance

Knowing the order of events can have profound impacts, for example in predicting the path of a natural disaster or picking the next best stock trade. InfoSphere Streams helps insurance companies plan for natural disasters and enables real-time public alerts. It also performs real-time analysis of sensor data collected from the Hudson River, one of the most instrumented bodies of water in the world. Check this out - https://www.youtube.com/watch?v=y3CZQOtVx6s&list=PLA98824D75176BAEB&index=18
  • Claims fraud detection
  • Next best action and customer retention
  • Catastrophe risk modeling
  • Usage-based insurance
  • Portfolio management
  • Producer optimization

Oil & Gas

  • Advanced condition monitoring
  • Drilling surveillance & optimization
  • Production surveillance & optimization

Retail

  • Merchandise optimization
  • Actionable customer insight

Telecommunications

Telecommunications service providers continue to experience a huge growth in smartphone and mobile device use. Growing text and data usage creates a deluge of context- and time-sensitive data. InfoSphere Streams enables telecommunications providers to analyze billions of call data records per day to detect fraud, ensure high asset utilization and create accurate customer profiles for heightened customer service and retention. Using InfoSphere Streams, Sprint reduced storage costs by 90 percent. Check this out - https://www.youtube.com/watch?v=eg8KSLAZ2HM&feature=player_embedded
  • Pro-active call center
  • Smarter campaigns
  • Network analytics
  • Location-based services

Travel & Transportation

  • Customer analytics and loyalty marketing
  • Capacity & pricing optimization
  • Predictive maintenance optimization
 Reference: