Thorsten Schöning MySQL doesn't shutdown cleanly sometimes
May 23, 2016; 11:39
MySQL doesn't shutdown cleanly sometimes
I have a fresh install of Ubuntu 16.04 LTS Server x64 in a VirtualBox VM with its provided MySQL, without any additional databases currently. Sometimes the shutdown of MySQL itself fails with the logs looking like the following:
> 2016-05-20T12:59:28.924585Z 0 [Note] InnoDB: FTS optimize thread exiting. > 2016-05-20T12:59:28.924752Z 0 [Note] InnoDB: Starting shutdown... > 2016-05-20T12:59:29.026037Z 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool > 2016-05-20T12:59:29.026394Z 0 [Note] InnoDB: Buffer pool(s) dump completed at 160520 14:59:29 > 2016-05-20T13:00:29.872168Z 0 [Note] InnoDB: Waiting for page_cleaner to finish flushing of buffer pool [...]
The last line is repeated all over again, but nothing seems to happen anymore. If the shutdown of MySQL is initiated using SystemD, the process is killed after 10 minutes, without SystemD e.g. using mysqladmin it just hangs and seem to wait forever, at least a lot longer than 10 minutes and the last sentence above is repeated all over. One line per minute.
I've already googled regarding this message but couldn't find anything that would resolve my problem: We didn't change any passwords from their defaults in SystemD, we didn't "optimize" threading, we have no I/O performance issue, there's exactly no load at all and MySQL itself says that there are no dirty pages, and the problem doesn't occur always, only pretty often. I guess around 1/3 of the time, in all other cases the shutdown is fast and successful.
The problem occurred with an earlier installation of 16.04 beta release and some databases loaded as well, so we decided to make a clean install with the release sources and no databases to rule out as much as possible. But now we have a somewhat default installation, the problem still occurs and none of the tips on the web helps.
Additionally I had a hard time finding some more detailed information about which files exactly are written and maybe even how, regarding threading, flushing, problems with underlying I/O (SAS vs. SCSI vs. SATA etc.).
Do you have any idea what else could be the problem or where I could have a look at?
Our only workaround currently is killing the process after giving it some time to flush, just like SystemD does, only that we don't want/need to wait 10 minutes.