KBEC-00235 - Removing duplicate jobIds to unjam the background job deleter

Description

You notice deleted jobs remain.

or

You notice Sentry jobs accumulating on the jobs page. Normally there is only one Sentry job as Sentry cleans up after itself.

and

You find messages of the following form in the commander.log

2011-07-07T16:56:34.597 | ERROR | Workflow-003         |   69528 | 1396873 | backgroundDelete                              | OperationInvoker               | Operation exception: org.hibernate.NonUniqueResultException: query did not return a unique result: 2org.hibernate.NonUniqueResultException: query did not return a unique result: 2

Solution

WARNING: This solution involves running queries in your Database. It is recommended that only DBA's or users with experience running queries use these solutions. 

The following solution can be done when ElectricFlow is running.

  1. Find the results of this SQL query:
    SELECT a.id,
           b.id,
           a.entity_id
    FROM   ec_session a,
           ec_session b
    WHERE  a.entity_type = 'jobStep'
           AND b.entity_type = 'jobStep'
           AND a.entity_id = b.entity_id
           AND a.id != b.id;
    
  2. The query result would look something like this:
    ID ID ENTITY_ID
    ---------------------- ---------------------- ----------------------
    6226119 6225025 13494960
    6225025 6226119 13494960
    6230676 6231067 13502403
    6231067 6230676 13502403
    
  3. Any number that appears in both columns is a duplicate and one of the two entries has to be removed. Pick one of the entries and use SQL to delete the duplicates. The following SQL code deletes the duplicates in the above example:
    DELETE FROM ec_session_auth WHERE session_id IN ( 6226119, 6231067 );
    DELETE FROM ec_session WHERE id IN ( 6226119, 6231067 );
    commit;
    

Explanation

There appears to be a race condition - more prevalent when ElectricFlow is restarted - that enters duplicate jobIds into the database.

Have more questions? Submit a request

Comments

Powered by Zendesk