Teradata\u00a0is a relational\u00a0database\u00a0management system (RDBMS) that is:<\/p>\n
- \n
- an open system, running on a UNIX MP-RAS or Windows server platform.<\/li>\n
- capable of supporting many concurrent users from various client platforms.<\/li>\n<\/ul>\n
<\/p>\n
What is Internet Of Things (IoT):<\/strong><\/p>\n
The\u00a0Internet of Things\u00a0(IoT) is a system of interrelated computing devices, mechanical and digital machines, objects, animals or people that are provided with unique identifiers and the ability to transfer data over a network without requiring human-to-human or human-to-computer interaction. (From the web)<\/p>\n
The Internet of Things is a big part of what makes big data so big.<\/em><\/p>\n
<\/p>\n
Why Teradata Listener?<\/strong><\/p>\n
Teradata Listener helps users \u201clisten\u201d to massive streams of IoT data in real time and then use analytics to find the distinctive underlying patterns.<\/p>\n
Teradata Listener is self-service software for ingesting and distributing individual or multiple data streams from sources including sensors, telematics, mobile events, click streams, social media feeds and IT server logs.<\/p>\n
![\"\"](\"https:\/\/techblog.xavient.com\/wp-content\/uploads\/2017\/09\/1.png\")
Teradata IOT capabilities<\/figcaption><\/figure>\n <\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
Teradata Listener is designed to:<\/strong><\/p>\n
- \n
- Provide a self-service solution for programmers that minimizes the complexity of building and supporting data streams<\/li>\n
- Operate as a central service to ingest and write 100s of streams<\/li>\n
- Reliably deliver data without loss<\/li>\n
- Provide low latency ingestion for near real-time applications<\/li>\n
- Run as a software only solution, in the data center or in the cloud<\/li>\n<\/ul>\n
<\/p>\n
Possibilities to deploy:<\/strong><\/p>\n
The software can be deployed on-premises or in the cloud. It can push data to Hadoop, Teradata Aster Analytics, Teradata Database or other platforms, where data scientists, business analysts and developers analyze it without having to rely on IT for help.<\/p>\n
Running on the platform, the Teradata Database offers access to data in formats including XML, name-value pair, BSON (Binary JSON) and JSON from Web applications, sensors, and Internet of Things-connected machines.<\/p>\n
![\"\"](\"https:\/\/techblog.xavient.com\/wp-content\/uploads\/2017\/09\/2..png\")
How listener works?<\/figcaption><\/figure>\n <\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
A software gateway that the company described as having \u201creal-time listening capabilities to follow multiple streams of sensor and IOT data and propagate it into multiple platforms.\u201d Based upon an assortment of open source tools that includes the Apache Kafka message broker, Apache Mesos cluster manager, Elastic Search search engine, OpenStack cloud platform, Docker containers and micro services, Listener is designed to ingest large volumes of real-time data and dispatch it to back-end databases, file systems and analytics engines for processing.<\/p>\n
It\u2019s frictionless; you can load analytics in real time, dump it into the database and then run analytics at scale, Users can invoke data streams themselves using API keys, and track and analyze metadata about streaming information to know things like source, type and volume of data.<\/p>\n
Listener is well-suited to feed an Aster engine by gathering data from many real-time streams \u2013 such as sensor readings, Twitter data, real-time stock feeds and Spark analytics \u2013 and feeding it into multiple back ends. Listener doesn\u2019t process or transform data but rather delivers it to back-end systems like Apache Spark, Tibco and IBM DB2 for processing. Spark can be used as effectively a pre-processor to perform top-level analysis before streaming data back into other data stores. A single stream can be sent to multiple repositories simultaneously.<\/p>\n
![\"\"](\"https:\/\/techblog.xavient.com\/wp-content\/uploads\/2017\/09\/3..png\")
Source system to Target systems data migration<\/figcaption><\/figure>\n <\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
Teradata Listener\u00a0Data Flow:<\/strong><\/p>\n
Listener is intelligent, self-service software for ingesting and distributing fast-moving data streams\u2013either individual or multiple streams\u2013at one time.\u00a0It allows customers to push data to Hadoop, Teradata Aster Analytics, Teradata Database, and other platforms.\u00a0It enables data scientists, business analysts, and developers to quickly and easily analyze new data streams for faster answers to business questions. Without needing to rely on IT for help, users can analyze data from numerous sources including sensors, telematics, mobile events, click streams, social media feeds, and IT server logs.<\/p>\n
![\"\"](\"https:\/\/techblog.xavient.com\/wp-content\/uploads\/2017\/09\/4..png\")
Ingestion process<\/figcaption><\/figure>\n <\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
Data Flow works in below three phases for Teradata Listener:<\/p>\n
- \n
- Real Time Streams<\/li>\n
- Collect & Distribute<\/li>\n
- Transformations & Analytics<\/li>\n<\/ol>\n
Listener\u2019s ingestion services are accessible by popular interfaces:<\/p>\n
- \n
- RESTful ingestion<\/strong>\u00a0\u2013 the very popular http transport protocol\u2014a universally accepted protocol for modern applications<\/li>\n
- MQTT<\/strong>\u00a0\u2013 the ideal protocol to overcome the emerging M2M or IoT world of connected devices where bandwidth and battery power are at a premium<\/li>\n<\/ul>\n
Any user can easily invoke the Listener\u2019s ingestion services to send continuous data streams to a data warehouse, discovery platform, or Hadoop platforms.<\/p>\n
A technology for self-service data ingest. Listener simplifies end-user (such as the application developer or marketing intelligence) and IT complexity by providing a single platform to deploy\u00a0and manage an end-user specified ingestion and distribution model, significantly reducing deployment time and cost of ownership.<\/p>\n
<\/p>\n
Listener Logical Architecture \u2013<\/strong><\/p>\n
![\"\"](\"https:\/\/techblog.xavient.com\/wp-content\/uploads\/2017\/09\/5.-1.jpg\")
Listener Architecture<\/figcaption><\/figure>\n <\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
Teradata listener can work with many cross platforms, Please have the below scenario 1:<\/p>\n
Teradata Listener Definition to Link Oracle with Teradata:<\/strong><\/p>\n
There are multiple ways to link Oracle with Teradata:<\/p>\n
- \n
- Oracle Gateway links to Teradata via a gateway listener<\/li>\n
- Heterogeneous Services offers an extra cost option for linking to Teradata<\/li>\n
- Golden Gate provides a link to Teradata<\/li>\n
- ODBC link with Teradata via ODBD<\/li>\n<\/ul>\n
Linking Oracle to Teradata via Oracle Gateway requires that a listener be defined:<\/p>\n
SID_LIST_LISTENER=\r\n\r\n(SID_LIST=\r\n\r\n(SID_DESC=.\r\n\r\n.\r\n\r\n)\r\n\r\n(SID_DESC=.\r\n\r\n.\r\n\r\n)\r\n\r\n(SID_DESC=\r\n\r\n(SID_NAME=gateway_sid)\r\n\r\n(ORACLE_HOME=oracle_home_directory)\r\n\r\n(PROGRAM=dg4tera) (ENVS=PATH=oracle_home_directory\\bin;teradata_client_directory\\lib;Windows_system_paths)\r\n\r\n)\r\n\r\n)<\/pre>\n
<\/p>\n
Teradata Listener Dashboard \u2013\u00a0<\/strong><\/p>\n
![\"\"](\"https:\/\/techblog.xavient.com\/wp-content\/uploads\/2017\/09\/6.-1.jpg\")
Dashboard<\/figcaption><\/figure>\n <\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
<\/p>\n
Teradata Listener Features:<\/strong><\/p>\n
- \n
- Enterprise wide platform for ingesting high volume real-time streams of data<\/strong><\/li>\n<\/ul>\n
Leverage a central platform across the enterprise for capturing and controlling large volumes of real-time streams from any source such as websites, IoT, social feeds, system and application logs.<\/p>\n
- \n
- Pre-built integration with Teradata UDA<\/strong><\/li>\n<\/ul>\n
Pre-integration with Teradata UDA allows users to easily persist the data in real-time or batches to Teradata, Aster or Hadoop.<\/p>\n
- \n
- Automatic Metrics Calculation<\/strong><\/li>\n<\/ul>\n
The intuitive dashboards on Teradata Listener provides understanding of the volume, latency health of your streaming pipelines at a glance.<\/p>\n
- \n
- Integrate with external apps<\/strong><\/li>\n<\/ul>\n
Easy to use APIs to develop custom processing of the real-time streams, build real-time streaming analytics, power real-time dashboards and generate alerts.<\/p>\n
- \n
- Self-service for users<\/strong><\/li>\n<\/ul>\n
Users no longer need to wait for IT to get data into their platform of choice. With Teradata Listener, any user in the enterprise can easily add data streams to Teradata Listener and persist the data in a location they have access to.<\/p>\n
- \n
- Fully supported and enterprise grade<\/strong><\/li>\n<\/ul>\n
Users do not need to understand the complexities of the various Open Source solutions used by Listener.<\/p>\n
- \n
- Microservice cloud architecture<\/strong><\/li>\n<\/ul>\n
Built for on-premises cloud or external cloud infrastructures. It is composed of individual micoservices, each with its own public API and placed in Docker containers so that any component of Listener can be scaled for performance or availability.<\/p>\n
- \n
- Modern UI & UX<\/strong><\/li>\n<\/ul>\n
Teradata Listener provides a simple user experience built on Material Design.<\/p>\n
Teradata Listener Business Use Cases:<\/strong><\/p>\n
- \n
- Website Activity<\/li>\n
- IoT\/Sensors<\/li>\n
- Email Compliance<\/li>\n
- Sales Spikes<\/li>\n
- Track Logistics<\/li>\n
- Track Social Media<\/li>\n
- Customer Satisfaction<\/li>\n
- Reservation Brokers<\/li>\n
- Recommendations<\/li>\n
- Security Data<\/li>\n<\/ul>\n
Teradata Listener Setup Requirements:<\/strong><\/p>\n
- \n
- Listener must be installed on an existing OpenStack environment
\n\u2022 Listener requires Openstack versions 2014.2.x (Juno) or 2015.1.x (Kilo).
\n\u2022 The OpenStack environment must have sufficient quota allocation of resources for Teradata Listener.
\n\u2022 The actual amount of resources will depend on the number and volume of data being streamed into Listener.
\n\u2022 At a minimum, Teradata Listener will need about 45 vCPUs, 90 GB of RAM, 520 GB of disk, a quota of at least 13 VM instances, and 13 floating IPs.
\n\u2022 The server initiating the installation must:
\n\u2013 Be a Linux server with Vagrant installed. A system running linux or OS X(10.10.4) with Vagrant installed.
\n\u2013 The server also needs network access to the Internet to download the Teradata Listener installation images and the OpenStack environment.
\n\u2022 The Teradata Listener web client supports all browsers with versions n-1; where n is the current browser version:
\n\u2013 Includes browsers are the latest versions of Chrome, Firefox, Internet Explorer, and Safari, and their previous versions. For instance with Internet Explorer, this includes IE11 & IE10.
\n\u2022 LDAP for user authentication
\n\u2022 SMTP for system emails (un-authenticated internal relay)
\n\u2022 Teradata Listener support the following targets for persisting the data:
\n\u2013 Teradata 14.x and higher
\n\u2013 Hadoop 2.4.x (CDH 5.2 and higher \/ HDP 2.1 and higher) \u2013 Sequence File format only.
\n\u2013 HBase 0.98<\/li>\n<\/ul>\nFor more around our Teradata series, follow us on dbtut!!<\/p>\n
Credits:\u00a0<\/em><\/strong><\/p>\n
http:\/\/dbtut.com<\/a><\/p>\n
Pankaj Chahar<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0<\/strong><\/em><\/p>\n
\nSource and References:\u00a0<\/strong><\/em>
\nhttp:\/\/www.teradata.com<\/a><\/em><\/p>\n
- Listener must be installed on an existing OpenStack environment
- Modern UI & UX<\/strong><\/li>\n<\/ul>\n
- Microservice cloud architecture<\/strong><\/li>\n<\/ul>\n
- Fully supported and enterprise grade<\/strong><\/li>\n<\/ul>\n
- Self-service for users<\/strong><\/li>\n<\/ul>\n
- Integrate with external apps<\/strong><\/li>\n<\/ul>\n
- Automatic Metrics Calculation<\/strong><\/li>\n<\/ul>\n
- Pre-built integration with Teradata UDA<\/strong><\/li>\n<\/ul>\n
- Enterprise wide platform for ingesting high volume real-time streams of data<\/strong><\/li>\n<\/ul>\n
- MQTT<\/strong>\u00a0\u2013 the ideal protocol to overcome the emerging M2M or IoT world of connected devices where bandwidth and battery power are at a premium<\/li>\n<\/ul>\n
- RESTful ingestion<\/strong>\u00a0\u2013 the very popular http transport protocol\u2014a universally accepted protocol for modern applications<\/li>\n