Please upgrade here. These earlier versions are no longer being updated and have security issues.
HackerOne users: Testing against this community violates our program's Terms of Service and will result in your bounty being denied.

Sphinx Plugin Cron Job Errors

meshugymeshugy Musician/Hacker ✭✭

Hello,

I followed the instructions included with this plugin and setup three cron job (main, delta, and stats.) They are running several times a day, but when I look at the log it's showing these errors for every attempt:

WARNING: failed to open pid_file '/var/log/searchd.pid'.
WARNING: indices NOT rotated.

Any idea what I can do to avoid these errors? As it stands, they are preventing the indexer from running. I can get it to work manually, but I'd prefer to have the cron jobs automating this task.

The FAQ with the plugin mentions turning off searchd if these errors occur. I can do that when manually running the indecer, but how do you do that with a cron job?

Thanks!

Michael

Comments

  • mcu_hqmcu_hq yippie ki-yay ✭✭✭
    edited January 2014

    I suspect it may have to do with permissions. Since it works fine when you manually do it, can you verify that the cron job is running with root privleldges or at least the same user level as when you manually index? For sanity, also check that the cron file has the right locations of the pid file.

  • meshugymeshugy Musician/Hacker ✭✭
    edited January 2014

    Ok, I set the cron jobs to run as sudo and it seems to be working now. Here is what my log file looks like now. Does this look correct?

    Only the Delta and Stats have been indexed so far...the Main won't be attempted till tonight.

    Sphinx 2.1.4-release (rel21-r4421)
    Copyright (c) 2001-2013, Andrew Aksyonoff
    Copyright (c) 2008-2013, Sphinx Technologies Inc (http://sphinxsearch.com)
    
    using config file '/usr/local/etc/sphinx.conf'...
    WARNING: key 'port' is deprecated in /usr/local/etc/sphinx.conf line 237; use 'listen' instead.
    indexing index 'vss_delta'...
    Sphinx 2.1.4-release (rel21-r4421)
    Copyright (c) 2001-2013, Andrew Aksyonoff
    Copyright (c) 2008-2013, Sphinx Technologies Inc (http://sphinxsearch.com)
    
    using config file '/usr/local/etc/sphinx.conf'...
    WARNING: key 'port' is deprecated in /usr/local/etc/sphinx.conf line 237; use 'listen' instead.
    indexing index 'vss_stats'...
    collected 2567 docs, 0.0 MB
    sorted 0.0 Mhits, 100.0% done
    total 2567 docs, 19288 bytes
    total 0.046 sec, 412242 bytes/sec, 54864.49 docs/sec
    total 3 reads, 0.000 sec, 20.1 kb/call avg, 0.0 msec/call avg
    total 10 writes, 0.000 sec, 11.9 kb/call avg, 0.0 msec/call avg
    rotating indices: successfully sent SIGHUP to searchd (pid=22482).
    collected 10 docs, 0.0 MB
    collected 0 attr values
    sorted 0.0 Mvalues, 100.0% done
    sorted 0.0 Mhits, 100.0% done
    total 10 docs, 4713 bytes
    total 0.111 sec, 42144 bytes/sec, 89.42 docs/sec
    total 5 reads, 0.000 sec, 1.0 kb/call avg, 0.0 msec/call avg
    total 10 writes, 0.000 sec, 1.3 kb/call avg, 0.0 msec/call avg
    rotating indices: successfully sent SIGHUP to searchd (pid=22482).
    
  • mcu_hqmcu_hq yippie ki-yay ✭✭✭

    Looks about right. Best way to tell is to create new content and then let cron run. Search for the new material after cron ran to see if it got indexed.

  • meshugymeshugy Musician/Hacker ✭✭

    @mcu_hq said:
    Looks about right. Best way to tell is to create new content and then let cron run. Search for the new material after cron ran to see if it got indexed.

    It seems to be running the cron files correctly, but when I search for new content it's not indexed. So somehow it's still not indexing correctly. If I manually use the command

    sudo indexer --all --rotate
    

    It indexes everything properly and everything is searchable right away. Any ideas what may be preventing the cron from working correctly?

    Here is my latest log file:

    Sphinx 2.1.4-release (rel21-r4421)
    Copyright (c) 2001-2013, Andrew Aksyonoff
    Copyright (c) 2008-2013, Sphinx Technologies Inc (http://sphinxsearch.com)
    
    using config file '/usr/local/etc/sphinx.conf'...
    WARNING: key 'port' is deprecated in /usr/local/etc/sphinx.conf line 237; use 'listen' instead.
    indexing index 'vss_main'...
    collected 66465 docs, 33.9 MB
    collected 0 attr values
    sorted 0.0 Mvalues, 100.0% done
    sorted 6.2 Mhits, 100.0% done
    total 66465 docs, 33899649 bytes
    total 9.308 sec, 3641840 bytes/sec, 7140.33 docs/sec
    total 8 reads, 0.022 sec, 2473.0 kb/call avg, 2.7 msec/call avg
    total 52 writes, 0.101 sec, 877.2 kb/call avg, 1.9 msec/call avg
    rotating indices: successfully sent SIGHUP to searchd (pid=22482).
    Sphinx 2.1.4-release (rel21-r4421)
    Copyright (c) 2001-2013, Andrew Aksyonoff
    Copyright (c) 2008-2013, Sphinx Technologies Inc (http://sphinxsearch.com)
    
    using config file '/usr/local/etc/sphinx.conf'...
    WARNING: key 'port' is deprecated in /usr/local/etc/sphinx.conf line 237; use 'listen' instead.
    indexing index 'vss_stats'...
    collected 3030 docs, 0.0 MB
    sorted 0.0 Mhits, 100.0% done
    total 3030 docs, 23001 bytes
    total 0.082 sec, 279029 bytes/sec, 36757.56 docs/sec
    total 3 reads, 0.000 sec, 23.8 kb/call avg, 0.0 msec/call avg
    total 10 writes, 0.000 sec, 14.1 kb/call avg, 0.0 msec/call avg
    rotating indices: successfully sent SIGHUP to searchd (pid=22482).
    Sphinx 2.1.4-release (rel21-r4421)
    Copyright (c) 2001-2013, Andrew Aksyonoff
    Copyright (c) 2008-2013, Sphinx Technologies Inc (http://sphinxsearch.com)
    
    using config file '/usr/local/etc/sphinx.conf'...
    WARNING: key 'port' is deprecated in /usr/local/etc/sphinx.conf line 237; use 'listen' instead.
    indexing index 'vss_delta'...
    collected 2 docs, 0.0 MB
    collected 0 attr values
    sorted 0.0 Mvalues, 100.0% done
    sorted 0.0 Mhits, 100.0% done
    total 2 docs, 503 bytes
    total 0.261 sec, 1922 bytes/sec, 7.64 docs/sec
    total 5 reads, 0.000 sec, 0.1 kb/call avg, 0.0 msec/call avg
    total 10 writes, 0.000 sec, 0.2 kb/call avg, 0.0 msec/call avg
    rotating indices: successfully sent SIGHUP to searchd (pid=22482).
    
  • mcu_hqmcu_hq yippie ki-yay ✭✭✭

    That Looks alright to me. If you compare the cron output to the manual method output, are there any differences?

  • meshugymeshugy Musician/Hacker ✭✭
    edited January 2014

    @mcu_hq there is no entry in the sphinx cron log when I just run the indexer manually.

  • meshugymeshugy Musician/Hacker ✭✭
    edited January 2014

    OK, i figured it out. The path to the vss files was not set correctly in the sphinx.conf file. For example, the path to the vss_main was set as

    index vss_main
    {
        source          = vss_main_comment
        source          = vss_main_discussion
        path            =  vss_main
    

    For the cron file to work, it had to be set to:

    index vss_main
    {
        source          = vss_main_comment
        source          = vss_main_discussion
        path            = /home/myusername/vss_main
    

    I set the vss_delta and vss_stats the same way and now everything is getting indexed via the cron file.

  • mcu_hqmcu_hq yippie ki-yay ✭✭✭
    edited January 2014

    Ahhhhh. Thanks for tracking it down. That should have been replaced by the installer. I'll upload a fix on github!

  • meshugymeshugy Musician/Hacker ✭✭

    @mcu_hq said:
    Ahhhhh. Thanks for tracking it down. That should have been replaced by the installer. I'll upload a fix on github!

    My pleasure, thanks for the awsome plugin!

Sign In or Register to comment.