{"id":1668,"date":"2018-08-14T09:22:32","date_gmt":"2018-08-14T09:22:32","guid":{"rendered":"http:\/\/dbtut.com\/?p=1668"},"modified":"2018-11-09T23:01:50","modified_gmt":"2018-11-09T23:01:50","slug":"log-file-maintenance-and-db-corruption","status":"publish","type":"post","link":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/","title":{"rendered":"PG Log File Maintenance and DB Corruption"},"content":{"rendered":"<p>&nbsp;<\/p>\n<ul>\n<li>It is a good idea to save the database server&#8217;s log output\u00a0somewhere, rather than just routing it to \/dev\/null. The\u00a0log output is invaluable when it comes time to diagnose\u00a0problems. However, the log output tends to be\u00a0voluminous (especially at higher debug levels) and you\u00a0won&#8217;t want to save it indefinitely. You need\u00a0to &#8220;rotate&#8221; the log files so that new log files are started\u00a0and old ones removed after a reasonable period of time.<\/li>\n<li>If you simply direct the stderr of postgres into a file, you\u00a0will have log output, but the only way to truncate the log\u00a0file is to stop and restart the server. This might be OK if\u00a0you are using PostgreSQL in a development environment,\u00a0but few production servers would find this behavior\u00a0acceptable.<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p><span style=\"text-decoration: underline;\"><strong>DB Corruption<\/strong><\/span><\/p>\n<ul style=\"list-style-type: circle;\">\n<li><strong>Causes<\/strong><\/li>\n<\/ul>\n<ul>\n<li>Underlying storage failure.<\/li>\n<li>Bad disk, bad controller.<\/li>\n<li>Garbage writes during power loss.<\/li>\n<li>Battery backup that didn\u2019t.<\/li>\n<li>Bad RAM.<\/li>\n<li>Hardware Failures<\/li>\n<li>PostgreSQL bugs.<\/li>\n<li>9.x had a series of unfortunate replication bugs.<\/li>\n<li>Used to be extremely rare.<\/li>\n<li>With luck, will become extremely rare again.<\/li>\n<li>Operator error.<\/li>\n<li>Backups that do not include critical files.<\/li>\n<li>Backups that do not follow protocol.<\/li>\n<li>Backups that forget external table spaces<\/li>\n<li>Bungled attempts at problem recovery.<\/li>\n<li>Delete the wrong files to free space<\/li>\n<\/ul>\n<ul style=\"list-style-type: circle;\">\n<li><strong>Prevention<\/strong><\/li>\n<\/ul>\n<ul>\n<li>Buy good hardware, demand your cloud provider do so, or have multi-tier\u00a0redundancy.<\/li>\n<li>Make backups, and test them.<\/li>\n<li>Stay up on PostgreSQL releases, and read the release notes.<\/li>\n<li>DR\/HA to be in place with cluster setup<\/li>\n<li>Proper backup strategy to be in place<\/li>\n<li>Pro-Active alerting may include third party tools to avoid any untoward incident<\/li>\n<\/ul>\n\n<div class=\"pvc_clear\"><\/div>\n<p id=\"pvc_stats_1668\" class=\"pvc_stats all  \" data-element-id=\"1668\" style=\"\"><i class=\"pvc-stats-icon medium\" aria-hidden=\"true\"><svg aria-hidden=\"true\" focusable=\"false\" data-prefix=\"far\" data-icon=\"chart-bar\" role=\"img\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\" class=\"svg-inline--fa fa-chart-bar fa-w-16 fa-2x\"><path fill=\"currentColor\" d=\"M396.8 352h22.4c6.4 0 12.8-6.4 12.8-12.8V108.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v230.4c0 6.4 6.4 12.8 12.8 12.8zm-192 0h22.4c6.4 0 12.8-6.4 12.8-12.8V140.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v198.4c0 6.4 6.4 12.8 12.8 12.8zm96 0h22.4c6.4 0 12.8-6.4 12.8-12.8V204.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v134.4c0 6.4 6.4 12.8 12.8 12.8zM496 400H48V80c0-8.84-7.16-16-16-16H16C7.16 64 0 71.16 0 80v336c0 17.67 14.33 32 32 32h464c8.84 0 16-7.16 16-16v-16c0-8.84-7.16-16-16-16zm-387.2-48h22.4c6.4 0 12.8-6.4 12.8-12.8v-70.4c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v70.4c0 6.4 6.4 12.8 12.8 12.8z\" class=\"\"><\/path><\/svg><\/i> <img loading=\"lazy\" decoding=\"async\" width=\"16\" height=\"16\" alt=\"Loading\" src=\"https:\/\/dbtut.com\/wp-content\/plugins\/page-views-count\/ajax-loader-2x.gif\" border=0 \/><\/p>\n<div class=\"pvc_clear\"><\/div>\n","protected":false},"excerpt":{"rendered":"<p>&nbsp; It is a good idea to save the database server&#8217;s log output\u00a0somewhere, rather than just routing it to \/dev\/null. The\u00a0log output is invaluable when it comes time to diagnose\u00a0problems. However, the log output tends to be\u00a0voluminous (especially at higher debug levels) and you\u00a0won&#8217;t want to save it indefinitely. You need\u00a0to &#8220;rotate&#8221; the log files &hellip;<\/p>\n<div class=\"pvc_clear\"><\/div>\n<p id=\"pvc_stats_1668\" class=\"pvc_stats all  \" data-element-id=\"1668\" style=\"\"><i class=\"pvc-stats-icon medium\" aria-hidden=\"true\"><svg aria-hidden=\"true\" focusable=\"false\" data-prefix=\"far\" data-icon=\"chart-bar\" role=\"img\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" viewBox=\"0 0 512 512\" class=\"svg-inline--fa fa-chart-bar fa-w-16 fa-2x\"><path fill=\"currentColor\" d=\"M396.8 352h22.4c6.4 0 12.8-6.4 12.8-12.8V108.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v230.4c0 6.4 6.4 12.8 12.8 12.8zm-192 0h22.4c6.4 0 12.8-6.4 12.8-12.8V140.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v198.4c0 6.4 6.4 12.8 12.8 12.8zm96 0h22.4c6.4 0 12.8-6.4 12.8-12.8V204.8c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v134.4c0 6.4 6.4 12.8 12.8 12.8zM496 400H48V80c0-8.84-7.16-16-16-16H16C7.16 64 0 71.16 0 80v336c0 17.67 14.33 32 32 32h464c8.84 0 16-7.16 16-16v-16c0-8.84-7.16-16-16-16zm-387.2-48h22.4c6.4 0 12.8-6.4 12.8-12.8v-70.4c0-6.4-6.4-12.8-12.8-12.8h-22.4c-6.4 0-12.8 6.4-12.8 12.8v70.4c0 6.4 6.4 12.8 12.8 12.8z\" class=\"\"><\/path><\/svg><\/i> <img loading=\"lazy\" decoding=\"async\" width=\"16\" height=\"16\" alt=\"Loading\" src=\"https:\/\/dbtut.com\/wp-content\/plugins\/page-views-count\/ajax-loader-2x.gif\" border=0 \/><\/p>\n<div class=\"pvc_clear\"><\/div>\n","protected":false},"author":109,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_uf_show_specific_survey":0,"_uf_disable_surveys":false,"footnotes":""},"categories":[5],"tags":[],"class_list":["post-1668","post","type-post","status-publish","format-standard","","category-postgres"],"aioseo_notices":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v24.9 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>PG Log File Maintenance and DB Corruption - Database Tutorials<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"PG Log File Maintenance and DB Corruption - Database Tutorials\" \/>\n<meta property=\"og:description\" content=\"&nbsp; It is a good idea to save the database server&#8217;s log output\u00a0somewhere, rather than just routing it to \/dev\/null. The\u00a0log output is invaluable when it comes time to diagnose\u00a0problems. However, the log output tends to be\u00a0voluminous (especially at higher debug levels) and you\u00a0won&#8217;t want to save it indefinitely. You need\u00a0to &#8220;rotate&#8221; the log files &hellip;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/\" \/>\n<meta property=\"og:site_name\" content=\"Database Tutorials\" \/>\n<meta property=\"article:published_time\" content=\"2018-08-14T09:22:32+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2018-11-09T23:01:50+00:00\" \/>\n<meta name=\"author\" content=\"Vaibhav Krishna\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Vaibhav Krishna\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"1 minute\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/\"},\"author\":{\"name\":\"Vaibhav Krishna\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/person\/6c10f95f66ae2b4026552b02018b18b7\"},\"headline\":\"PG Log File Maintenance and DB Corruption\",\"datePublished\":\"2018-08-14T09:22:32+00:00\",\"dateModified\":\"2018-11-09T23:01:50+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/\"},\"wordCount\":280,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/dbtut.com\/#organization\"},\"articleSection\":[\"PostgreSQL\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/\",\"url\":\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/\",\"name\":\"PG Log File Maintenance and DB Corruption - Database Tutorials\",\"isPartOf\":{\"@id\":\"https:\/\/dbtut.com\/#website\"},\"datePublished\":\"2018-08-14T09:22:32+00:00\",\"dateModified\":\"2018-11-09T23:01:50+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/dbtut.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"PG Log File Maintenance and DB Corruption\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/dbtut.com\/#website\",\"url\":\"https:\/\/dbtut.com\/\",\"name\":\"Database Tutorials\",\"description\":\"MSSQL, Oracle, PostgreSQL, MySQL, MariaDB, DB2, Sybase, Teradata, Big Data, NOSQL, MongoDB, Couchbase, Cassandra, Windows, Linux\",\"publisher\":{\"@id\":\"https:\/\/dbtut.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/dbtut.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/dbtut.com\/#organization\",\"name\":\"dbtut\",\"url\":\"https:\/\/dbtut.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg\",\"contentUrl\":\"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg\",\"width\":223,\"height\":36,\"caption\":\"dbtut\"},\"image\":{\"@id\":\"https:\/\/dbtut.com\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/person\/6c10f95f66ae2b4026552b02018b18b7\",\"name\":\"Vaibhav Krishna\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/dbtut.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/482ae666004473e37e849efeedbf7111152f716f6e9fc6852074d49536796697?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/482ae666004473e37e849efeedbf7111152f716f6e9fc6852074d49536796697?s=96&d=mm&r=g\",\"caption\":\"Vaibhav Krishna\"},\"url\":\"https:\/\/dbtut.com\/index.php\/author\/vaibhavkrishna\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"PG Log File Maintenance and DB Corruption - Database Tutorials","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/","og_locale":"en_US","og_type":"article","og_title":"PG Log File Maintenance and DB Corruption - Database Tutorials","og_description":"&nbsp; It is a good idea to save the database server&#8217;s log output\u00a0somewhere, rather than just routing it to \/dev\/null. The\u00a0log output is invaluable when it comes time to diagnose\u00a0problems. However, the log output tends to be\u00a0voluminous (especially at higher debug levels) and you\u00a0won&#8217;t want to save it indefinitely. You need\u00a0to &#8220;rotate&#8221; the log files &hellip;","og_url":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/","og_site_name":"Database Tutorials","article_published_time":"2018-08-14T09:22:32+00:00","article_modified_time":"2018-11-09T23:01:50+00:00","author":"Vaibhav Krishna","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Vaibhav Krishna","Est. reading time":"1 minute"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#article","isPartOf":{"@id":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/"},"author":{"name":"Vaibhav Krishna","@id":"https:\/\/dbtut.com\/#\/schema\/person\/6c10f95f66ae2b4026552b02018b18b7"},"headline":"PG Log File Maintenance and DB Corruption","datePublished":"2018-08-14T09:22:32+00:00","dateModified":"2018-11-09T23:01:50+00:00","mainEntityOfPage":{"@id":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/"},"wordCount":280,"commentCount":0,"publisher":{"@id":"https:\/\/dbtut.com\/#organization"},"articleSection":["PostgreSQL"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/","url":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/","name":"PG Log File Maintenance and DB Corruption - Database Tutorials","isPartOf":{"@id":"https:\/\/dbtut.com\/#website"},"datePublished":"2018-08-14T09:22:32+00:00","dateModified":"2018-11-09T23:01:50+00:00","breadcrumb":{"@id":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/dbtut.com\/index.php\/2018\/08\/14\/log-file-maintenance-and-db-corruption\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/dbtut.com\/"},{"@type":"ListItem","position":2,"name":"PG Log File Maintenance and DB Corruption"}]},{"@type":"WebSite","@id":"https:\/\/dbtut.com\/#website","url":"https:\/\/dbtut.com\/","name":"Database Tutorials","description":"MSSQL, Oracle, PostgreSQL, MySQL, MariaDB, DB2, Sybase, Teradata, Big Data, NOSQL, MongoDB, Couchbase, Cassandra, Windows, Linux","publisher":{"@id":"https:\/\/dbtut.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/dbtut.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/dbtut.com\/#organization","name":"dbtut","url":"https:\/\/dbtut.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/dbtut.com\/#\/schema\/logo\/image\/","url":"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg","contentUrl":"https:\/\/dbtut.com\/wp-content\/uploads\/2021\/02\/dbtutlogo.jpg","width":223,"height":36,"caption":"dbtut"},"image":{"@id":"https:\/\/dbtut.com\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/dbtut.com\/#\/schema\/person\/6c10f95f66ae2b4026552b02018b18b7","name":"Vaibhav Krishna","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/dbtut.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/482ae666004473e37e849efeedbf7111152f716f6e9fc6852074d49536796697?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/482ae666004473e37e849efeedbf7111152f716f6e9fc6852074d49536796697?s=96&d=mm&r=g","caption":"Vaibhav Krishna"},"url":"https:\/\/dbtut.com\/index.php\/author\/vaibhavkrishna\/"}]}},"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/posts\/1668","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/users\/109"}],"replies":[{"embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/comments?post=1668"}],"version-history":[{"count":0,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/posts\/1668\/revisions"}],"wp:attachment":[{"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/media?parent=1668"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/categories?post=1668"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dbtut.com\/index.php\/wp-json\/wp\/v2\/tags?post=1668"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}