{"id":11939,"date":"2019-06-02T21:32:05","date_gmt":"2019-06-02T21:32:05","guid":{"rendered":"https:\/\/dbtut.com\/?p=11939"},"modified":"2019-07-23T13:48:56","modified_gmt":"2019-07-23T13:48:56","slug":"how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash","status":"publish","type":"post","link":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/","title":{"rendered":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash"},"content":{"rendered":"

In this article we will transfer the following data set from the AdventureWorks database, which is the sample database of SQL Server, to logstash with elasticsearch. If you want to move CSV or TXT File to Elasticsearch you can read the below article.<\/p>\n

“How To Move CSV or TXT To Elasticsearch<\/a>”<\/p>\n

Source Query in AdventureWorks2014 Database<\/h2>\n
SELECT [ProductID] --int\n      ,[Name] --nvarchar(25)    \n\t  ,CASE WHEN [Color] is null THEN 'Uncoloured' ELSE [Color] END AS [Color] --nvarchar(15)\n      ,[SafetyStockLevel] --smallint\n      ,[ListPrice] --money\n      ,CASE WHEN [Weight] is null THEN 0 ELSE CONVERT(float,[Weight]) END AS [Weight] --decimal(8,2)  to float--There will be an error if you do not convert.\n      ,SUBSTRING(CONVERT(varchar(50), [SellStartDate],120),0,11) AS [SellStartDate] --datetime\n  FROM [AdventureWorks2014].[Production].[Product]\n<\/pre>\n
\"\"<\/p>\n
Create Elasticsearch Index With Mapping<\/h2>\n
If we transfer data from MS SQL without mapping in elasticsearch, all data is transferred as text. To transfer data with the correct data types, we need to mapping in elasticsearch.<\/p>\n
According to the above data types, we create an index in elasticsearch as follows. You should note the replica and shard numbers. We have not specified in our example. You may want to read the following article for more information.<\/p>\n
“How To Create an Elasticsearch Indice using Kibana<\/a>”<\/p>\n
PUT my_index_mssql\n  {\n    \"mappings\": {\n      \"doc\": {\n        \"properties\": {\n          \"@timestamp\": {\n            \"type\": \"date\"\n          },\n          \"@version\": {\n            \"type\": \"text\",\n            \"fields\": {\n              \"keyword\": {\n                \"type\": \"keyword\",\n                \"ignore_above\": 256\n              }\n            }\n          },\n          \"ProductID\": {\n            \"type\": \"integer\"\n          },\n          \"Name\": {\n            \"type\": \"text\",\n            \"fields\": {\n              \"keyword\": {\n                \"type\": \"keyword\",\n                \"ignore_above\": 25\n              }\n            }\n          },\n          \"Color\": {\n            \"type\": \"text\",\n            \"fields\": {\n              \"keyword\": {\n                \"type\": \"keyword\",\n                \"ignore_above\": 15\n              }\n            }\n          },\n          \"SafetyStockLevel\": {\n            \"type\": \"short\"\n          },   \n          \"ListPrice\": {\n            \"type\": \"float\"\n          },\n          \"Weight\": {\n            \"type\": \"float\"\n          },\n\t\t   \"SellStartDate\": {\n               \"type\":\"date\",\n               \"format\":\"YYYY-MM-dd\"\n          }\n        }\n      }\n    }\n  }<\/pre>\n
We can see the mapping of the index we created with the command below.<\/p>\n
GET my_index_mssql\/_mapping<\/pre>\n
\"\"<\/p>\n
In order to transfer the data from a source with logstash to Elasticsearch, we need to prepare a conf file on the server where logstash is installed.<\/p>\n
Download SQL JDBC Driver<\/h2>\n
We should download SQL JDBC Driver before configuring the .conf file.<\/p>\n
https:\/\/www.microsoft.com\/en-us\/download\/details.aspx?id=54671<\/a><\/p>\n
\"\"<\/p>\n
After downloading the file, we copy it to the server where logstash is installed. I copied it to \/home\/elasticsearch using xshell’s file transfer.<\/p>\n
\"\"<\/p>\n
Then go to the \/home\/elasticsearch directory using the following command.<\/p>\n
cd \/home\/elastic\/<\/pre>\n
Extract the file.<\/p>\n
tar -xzf sqljdbc_4.2.8112.200_enu.tar.gz<\/pre>\n
\"\"<\/p>\n
Configure the .conf File on the Server Where Logstash is Installed<\/h2>\n
Create the conf file in the \/etc\/logstash\/conf.d\/ directory as follows, and save it as mssql_dataset.conf. This directory may change in later versions. You should find the “conf.d” directory and create the .conf file in this directory.<\/p>\n
cd \/etc\/logstash\/conf.d\/<\/pre>\n
vi mssql_dataset.conf<\/pre>\n
\"\"<\/p>\n
Explanation of mssql_dataset.conf<\/h3>\n
\n\n\n\n\n\n\n\n\n\n\n
\n
jdbc_driver_library<\/p>\n<\/td>\n
\n
Find sqljdbc42.jar in the file we extracted above and write this directory<\/p>\n<\/td>\n<\/tr>\n
\n
jdbc_connection_string<\/p>\n<\/td>\n
\n
write your ip, port, username that will connect to SQL Server, and user password<\/p>\n<\/td>\n<\/tr>\n
\n
jdbc_user<\/p>\n<\/td>\n
\n
username that will connect to SQL Server<\/p>\n<\/td>\n<\/tr>\n
\n
jdbc_password<\/p>\n<\/td>\n
\n
password of the user<\/p>\n<\/td>\n<\/tr>\n
\n
statement<\/p>\n<\/td>\n
\n
sql query<\/p>\n<\/td>\n<\/tr>\n
\n
hosts<\/p>\n<\/td>\n
\n
write logstash ip<\/p>\n<\/td>\n<\/tr>\n
\n
user => elasticsearch_authorized_user<\/p>\n
password => elasticsearch_authorized_user_password<\/p>\n<\/td>\n
\n
We can Secure Elasticsearch using Search Guard. If you installed Search Guard, write the elasticsearch user and password in here.<\/p>\n<\/td>\n<\/tr>\n
\n
document_id<\/p>\n<\/td>\n
\n
This column must be unique for constantly transferring changed data to elasticsearch. If you do not define this parameter, logstash can not transfer changed data.<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Content of mssql_dataset.conf<\/h3>\n
input {\n\tjdbc {\n\t\tjdbc_driver_library => \"\/home\/elastic\/sqljdbc_4.2\/enu\/jre8\/sqljdbc42.jar\"\n\t\tjdbc_driver_class => \"com.microsoft.sqlserver.jdbc.SQLServerDriver\"\n\t\tjdbc_connection_string =>\"jdbc:sqlserver:\/\/172.14.243.23:1434;user=logstashUser;password=myfunnypassword;\"\n\t\tjdbc_user => \"logstashUser\"\n\t\tjdbc_password => \"myfunnypassword\"\n\t\tschedule => \"* * * * *\" --works every one minute. This works like crontab.\n\t\tstatement => \"SELECT [ProductID] --int\n\t\t\t\t\t\t,[Name] --nvarchar(25)    \n\t\t\t\t\t\t,CASE WHEN [Color] is null THEN 'Uncoloured' ELSE [Color] END AS [Color] --nvarchar(15)\n\t\t\t\t\t\t,[SafetyStockLevel] --smallint\n\t\t\t\t\t\t,[ListPrice] --money\n\t\t\t\t\t\t,CASE WHEN [Weight] is null THEN 0 ELSE [Weight] END AS [Weight] --decimal(8,2)  \n\t\t\t\t\t\t,SUBSTRING(CONVERT(varchar(50), [SellStartDate],120),0,11) AS [SellStartDate] --datetime\n\t\t\t\t\t  FROM [AdventureWorks2014].[Production].[Product]\" \n\t\ttype=>\"doc\" #our type name is doc. Look at mapping script\n}\n}\noutput {\n\telasticsearch {\n\t\thosts => \"172.14.243.24\"\n\t\tindex => \"my_index_mssql\" # we write our index name in here.\n\t\tdocument_id => \"%{productid}\" # use lower case.\n\t\tuser => elasticsearch_authorized_user\n\t\tpassword => elasticsearch_authorized_user_password\t\t\t\t\n\t}\n}\n<\/pre>\n
We paste the above commands into the conf file.<\/p>\n
Start Logstash and Transferring Data<\/h2>\n
Then I start the transfer with the help of the following command.<\/p>\n
\/usr\/share\/logstash\/bin\/logstash -f \/etc\/logstash\/conf.d\/mssql_dataset.conf --path.data=\/etc\/logstash\/ --path.settings=\/etc\/logstash\/<\/pre>\n
Because we schedule logstash to run every minute, it will check the records as follows.<\/p>\n
\"\"<\/p>\n
If you close the session, logstash will stop working. You can run the above script to start the logstash again. Also you can run this script with nohup or you can use screen method.<\/p>\n
nohup:<\/strong><\/p>\n
nohup \/usr\/share\/logstash\/bin\/logstash -f \/etc\/logstash\/conf.d\/mssql_dataset.conf --path.data=\/etc\/logstash\/ --path.settings=\/etc\/logstash\/ &<\/pre>\n
When I run the above command and check the nohup.out, I saw the below error.<\/p>\n
cat nohup.out<\/pre>\n
Logstash could not be started because there is already another instance using the configured data directory.\u00a0 If you wish to run multiple instances, you must change the “path.data” setting.<\/p>\n
\"\"<\/p>\n
Close the all sessions and run the command with nohup or find the logstash sessions and kill.<\/p>\n
Then you can run the command with nohup without error.<\/p>\n
ps -ef|grep logstash\nkill -9 number_of_logstash_pid<\/pre>\n
\"\"<\/p>\n
How To See Tranferred Data on Kibana<\/h2>\n
Let’s see our data through the kibana by creating an index pattern.<\/p>\n
We are connecting to Kibana to create an index pattern and click on discover. You can install Kibana by using the following article.<\/p>\n
“How To Install Kibana On Centos<\/a>”<\/p>\n
Go to the Management tab and click “Create index pattern”.<\/p>\n
\"\"<\/p>\n
Write the name of your index and click next in the index pattern section.<\/p>\n
\"\"<\/p>\n
On the next screen, we select a date field from the “Time filter field name” section. Data can be filtered according to the date field we selected here. Timestamp means the date on which data is transferred to elasticsearch.<\/p>\n
Finally, we click on Create Index Pattern.<\/p>\n
\"\"<\/p>\n
After creating the Index Pattern, we can see our data by selecting the date range as below.<\/p>\n
\"\"<\/p>\n
Or we can put filters as below.<\/p>\n
\"\"<\/p>\n\n<\/div>\n
<\/div>\n
<\/path><\/svg><\/i> \"Loading\"<\/p>\n
<\/div>\n","protected":false},"excerpt":{"rendered":"
In this article we will transfer the following data set from the AdventureWorks database, which is the sample database of SQL Server, to logstash with elasticsearch. If you want to move CSV or TXT File to Elasticsearch you can read the below article. “How To Move CSV or TXT To Elasticsearch” Source Query in AdventureWorks2014 …<\/p>\n
<\/div>\n
<\/path><\/svg><\/i> \"Loading\"<\/p>\n
<\/div>\n","protected":false},"author":1,"featured_media":11974,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[3654,3652],"tags":[3745,3751,3753,3752,3755,3748,3750,3747,3743,3749,3744,3746,3754],"class_list":["post-11939","post","type-post","status-publish","format-standard","has-post-thumbnail","","category-analyticdashboard","category-elk","tag-bulk-data-import-from-ms-sql-server-to-elasticsearch-using-logstash","tag-connecting-logstash-to-sql-server","tag-data-transfer-from-elasticsearch-to-sql-server","tag-elasticsearch-jdbc","tag-elasticsearch-logstash-sql-server","tag-elasticsearch-sql-server-integration","tag-export-data-from-sql-to-elasticsearch","tag-how-to-configure-logstash-to-transfer-data-from-ms-sql-server-to-elasticsearch","tag-how-to-copy-sql-server-data-to-elasticsearch-using-logstash","tag-import-data-from-sql-to-elasticsearch","tag-pushing-relational-data-to-elasticsearch-using-logstash","tag-sql-server-sync-with-elasticsearch-through-logstash","tag-sql-to-elasticsearch-using-logstash"],"aioseo_notices":[],"yoast_head":"\nHow To Transfer Data From MSSQL To Elasticsearch Using Logstash - Database Tutorials<\/title>\n<meta name=\"description\" content=\"How To Transfer Data From MSSQL To Elasticsearch Using Logstash\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How To Transfer Data From MSSQL To Elasticsearch Using Logstash - Database Tutorials\" \/>\n<meta property=\"og:description\" content=\"How To Transfer Data From MSSQL To Elasticsearch Using Logstash\" \/>\n<meta property=\"og:url\" content=\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/\" \/>\n<meta property=\"og:site_name\" content=\"Database Tutorials\" \/>\n<meta property=\"article:published_time\" content=\"2019-06-02T21:32:05+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2019-07-23T13:48:56+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png\" \/>\n\t<meta property=\"og:image:width\" content=\"582\" \/>\n\t<meta property=\"og:image:height\" content=\"374\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"dbtut\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"dbtut\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/\"},\"author\":{\"name\":\"dbtut\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/person\/fc047c39e1e53dce28fc4253529ea408\"},\"headline\":\"How To Transfer Data From MSSQL To Elasticsearch Using Logstash\",\"datePublished\":\"2019-06-02T21:32:05+00:00\",\"dateModified\":\"2019-07-23T13:48:56+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/\"},\"wordCount\":723,\"commentCount\":3,\"publisher\":{\"@id\":\"https:\/\/dbtut.com\/#organization\"},\"image\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png\",\"keywords\":[\"Bulk data import from MS SQL Server to Elasticsearch using logstash\",\"connecting logstash to sql server\",\"data transfer from elasticsearch to sql server\",\"elasticsearch jdbc\",\"elasticsearch logstash sql server\",\"elasticsearch sql server integration\",\"export data from sql to elasticsearch\",\"How to configure logstash to transfer data from MS-SQL server to Elasticsearch\",\"How to copy SQL Server data to Elasticsearch using LogStash\",\"import data from sql to elasticsearch\",\"Pushing Relational Data to Elasticsearch using Logstash\",\"SQL Server Sync with Elasticsearch through Logstash\",\"sql to elasticsearch using logstash\"],\"articleSection\":[\"Analytic&&Dashboard\",\"ELK\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/\",\"url\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/\",\"name\":\"How To Transfer Data From MSSQL To Elasticsearch Using Logstash - Database Tutorials\",\"isPartOf\":{\"@id\":\"https:\/\/dbtut.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png\",\"datePublished\":\"2019-06-02T21:32:05+00:00\",\"dateModified\":\"2019-07-23T13:48:56+00:00\",\"description\":\"How To Transfer Data From MSSQL To Elasticsearch Using Logstash\",\"breadcrumb\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage\",\"url\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png\",\"contentUrl\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png\",\"width\":582,\"height\":374},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/dbtut.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How To Transfer Data From MSSQL To Elasticsearch Using Logstash\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/dbtut.com\/#website\",\"url\":\"https:\/\/dbtut.com\/\",\"name\":\"Database Tutorials\",\"description\":\"MSSQL, Oracle, PostgreSQL, MySQL, MariaDB, DB2, Sybase, Teradata, Big Data, NOSQL, MongoDB, Couchbase, Cassandra, Windows, Linux\",\"publisher\":{\"@id\":\"https:\/\/dbtut.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/dbtut.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/dbtut.com\/#organization\",\"name\":\"dbtut\",\"url\":\"https:\/\/dbtut.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg\",\"contentUrl\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg\",\"width\":223,\"height\":36,\"caption\":\"dbtut\"},\"image\":{\"@id\":\"https:\/\/dbtut.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/person\/fc047c39e1e53dce28fc4253529ea408\",\"name\":\"dbtut\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c322c32021bf651d9e103b183963c479a9c9791ead0715f4348203496c39aa54?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c322c32021bf651d9e103b183963c479a9c9791ead0715f4348203496c39aa54?s=96&d=mm&r=g\",\"caption\":\"dbtut\"},\"description\":\"We are a team with over 10 years of database management and BI experience. Our Expertises: Oracle, SQL Server, PostgreSQL, MySQL, MongoDB, Elasticsearch, Kibana, Grafana.\",\"sameAs\":[\"http:\/\/NurullahCAKIR\"],\"url\":\"https:\/\/dbtut.com\/index.php\/author\/dbtut\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash - Database Tutorials","description":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/","og_locale":"en_US","og_type":"article","og_title":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash - Database Tutorials","og_description":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash","og_url":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/","og_site_name":"Database Tutorials","article_published_time":"2019-06-02T21:32:05+00:00","article_modified_time":"2019-07-23T13:48:56+00:00","og_image":[{"width":582,"height":374,"url":"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png","type":"image\/png"}],"author":"dbtut","twitter_card":"summary_large_image","twitter_misc":{"Written by":"dbtut","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#article","isPartOf":{"@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/"},"author":{"name":"dbtut","@id":"https:\/\/dbtut.com\/#\/schema\/person\/fc047c39e1e53dce28fc4253529ea408"},"headline":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash","datePublished":"2019-06-02T21:32:05+00:00","dateModified":"2019-07-23T13:48:56+00:00","mainEntityOfPage":{"@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/"},"wordCount":723,"commentCount":3,"publisher":{"@id":"https:\/\/dbtut.com\/#organization"},"image":{"@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage"},"thumbnailUrl":"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png","keywords":["Bulk data import from MS SQL Server to Elasticsearch using logstash","connecting logstash to sql server","data transfer from elasticsearch to sql server","elasticsearch jdbc","elasticsearch logstash sql server","elasticsearch sql server integration","export data from sql to elasticsearch","How to configure logstash to transfer data from MS-SQL server to Elasticsearch","How to copy SQL Server data to Elasticsearch using LogStash","import data from sql to elasticsearch","Pushing Relational Data to Elasticsearch using Logstash","SQL Server Sync with Elasticsearch through Logstash","sql to elasticsearch using logstash"],"articleSection":["Analytic&&Dashboard","ELK"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/","url":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/","name":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash - Database Tutorials","isPartOf":{"@id":"https:\/\/dbtut.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage"},"image":{"@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage"},"thumbnailUrl":"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png","datePublished":"2019-06-02T21:32:05+00:00","dateModified":"2019-07-23T13:48:56+00:00","description":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash","breadcrumb":{"@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#primaryimage","url":"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png","contentUrl":"https:\/\/dbtut.com\/wp-content\/uploads\/2019\/04\/Ads\u0131z-66.png","width":582,"height":374},{"@type":"BreadcrumbList","@id":"https:\/\/dbtut.com\/index.php\/2019\/06\/02\/how-to-transfer-data-from-mssql-to-elasticsearch-using-logstash\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/dbtut.com\/"},{"@type":"ListItem","position":2,"name":"How To Transfer Data From MSSQL To Elasticsearch Using Logstash"}]},{"@type":"WebSite","@id":"https:\/\/dbtut.com\/#website","url":"https:\/\/dbtut.com\/","name":"Database Tutorials","description":"MSSQL, Oracle, PostgreSQL, MySQL, MariaDB, DB2, Sybase, Teradata, Big Data, NOSQL, MongoDB, Couchbase, Cassandra, Windows, Linux","publisher":{"@id":"https:\/\/dbtut.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/dbtut.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/dbtut.com\/#organization","name":"dbtut","url":"https:\/\/dbtut.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/dbtut.com\/#\/schema\/logo\/image\/","url":"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg","contentUrl":"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg","width":223,"height":36,"caption":"dbtut"},"image":{"@id":"https:\/\/dbtut.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/dbtut.com\/#\/schema\/person\/fc047c39e1e53dce28fc4253529ea408","name":"dbtut","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/dbtut.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/c322c32021bf651d9e103b183963c479a9c9791ead0715f4348203496c39aa54?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c322c32021bf651d9e103b183963c479a9c9791ead0715f4348203496c39aa54?s=96&d=mm&r=g","caption":"dbtut"},"description":"We are a team with over 10 years of database management and BI experience. Our Expertises: Oracle, SQL Server, PostgreSQL, MySQL, MongoDB, Elasticsearch, Kibana, Grafana.","sameAs":["http:\/\/NurullahCAKIR"],"url":"https:\/\/dbtut.com\/index.php\/author\/dbtut\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/posts\/11939","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/comments?post=11939"}],"version-history":[{"count":0,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/posts\/11939\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/media\/11974"}],"wp:attachment":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/media?parent=11939"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/categories?post=11939"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/tags?post=11939"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}