KBEC-00104 - Signs you are out of disk space and remedies to recover disk space

Background

The disk system can fill up over time. Eventually, Flow will fail when the disk fills up.

Symptoms

After days of ElectricFlow running uninterrupted, one of the following occurs:

1. Users cannot login to the system.

2. When restarting Flow, Flow cannot connect to the database and reports in the GUI:

    

   Server not running: Waiting for a valid database configuration - go to the Database Configuration page

    

   or

    

   Server not running: Can't connect to the database: No suitable driver - go to the Database Configuration page

    

3. The commander.log file is no longer growing.
   1. Flow stopped writing to the log file when the disk became full.
   2. last error can be: "org.hibernate.exception.GenericJDBCException: Could not execute JDBC batch update"
4. Steps fail with "AGENT_INTERNAL_ERROR"
5. If you are using MySQL, stop and start MySQL - if MySQL is out of space, it will not restart.

Diagnosis

1. Check the Flow server's disk space. The Flow server log files have filled up the disk - look at the size in the "logs" Flow directory.
2. Check the database server's disk space. When using local MySQL, do the following:
   1. Look at the size of the files in the MySQL data directory.
   2. Search the MySQL error log, found in mysql/data/<node>.err by default, for any recent errors.

Remedy

1. Stop Flow.
2. If using MySQL, stop MySQL.
3. Recover disk space by compressing or deleting Flow log files.
4. Check the contents of file database.properties found in the Flow conf directory.
   1. This file can be left corrupted, for example, it has only two lines instead of the twelve normally in the file.
   2. Restore this file from backup.
5. If using MySQL, restart MySQL.
6. Restart Flow.