How to handle Heritrix stale file handle exception
Heritrix is the Internet Archive-s web archival software, essentially a web crawling bot that takes a list of web sites, and saves them as ARC/WARC files in order to create a web archive like the one at archive.org.
Sometimes, like every other piece of software, it can produce error messages that might not be trivial.
One of them is the following:
Caused by: java.nio.file.FileSystemException: /path/to/file: Stale file handle
Other than the exception, you might face the following problems:
- The REST API returns empty responses for certain jobs, instead of their status.
- The web UI shows a long chain of exception (including Stale file handle FileSystemException as the root cause) when navigating to the job’s status page
Cause:
One possible cause this issue is that Heritrix has a file open that is on a remote filesystem, and during Heritrix’s run the connection to that filesystem broke due to a network outage for example.
Solution:
- Safely shut down Heritrix’s other jobs ( pause, checkpoint )
- Restart Heritrix
After the restart if you continue the jobs they will be fine, and the error is gone.
Remote debugging Tomcat7 servlets with Netbeans
At work we still use Tomcat 7 in production and I needed to set up debugging for various development systems. This article shows how to enable Tomcat 7 remote debugging
Enabling Tomcat 7 remote debugging via JDWP
Linux
I use Ubuntu 16.04 LTE so I’ll use that in the example, but other distros will not be that much different, except for the path and (re)starting the service of course.
- Edit or create the file /usr/share/tomcat7/bin/setenv.sh and put in the following content:
export JAVA_OPTS=”-Xdebug \
-Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n”Note: Obviously if the file already exists and it already has some content, then just add the parameters instead of adding the entire line.
- Restart Tomcat
sudo service tomcat7 restart
Windows
- Go to the Tomcat binary directory, which is by default
c:\Program Files\Apache Software Foundation\Tomcat 7.0\bin
- Start the program Tomcat7w.exe
- Switch to the java tab and add the following lines to the Java options textbox:
-Xdebug
-Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=nNote: It is important that each of the parameters should be added on separate lines, and that lines should have no whitespaces in the end!
- Restarts Tomcat 7
net stop tomcat7
net start tomcat7
Attaching Netbeans debugger to Tomcat 7
Now that we have Tomcat running with the remote debugging on we can attach Netbeans to debug.
- Click debug – attach debugger, a dialog box will appear
- Select Java Debugger (JPDA) as the Debugger
- Select SocketAttach as Connector
- Fill in host / IP address to the host field
- Fill in port to the Port field, in this example the port is 8787, but obviously it can be any non-taken port
- Click OK
- If everything went OK the debugging tab should show up showing the running threads
…and that’s it! Happy bug hunting!
Diagrams with draw.io
Right now I am writing up network / system design paper for school, and I desperately needed a diagram tool, that I can use on any platform, and it can create pretty network diagrams. Guess what I’ve found a really good one!
( NOTE: The picture is NOT my actual diagram, just a sketch )
Broken youtube account
So I accidentally broke my youtube account some time ago: I deleted my Google+ account, and while the youtube account remained, I couldn’t upload anymore! ( Except for email uploads ).
I didn’t know what to do, so I wanted to create a new youtube account.
My first idea was to create a new youtube page and use the youtube channel of that.
Unfortunately that didn’t work!
So I ended up creating a new Google account, a new Google+ account for that account, and transfer the broken page to it, then transfer it back to the original account. That fixed it!
I read an article not long ago that Google is like the old Microsoft now. Regarding this one particular issue, it’s definitely true! I just had to power off and on again, and the problem went away! 😀
In my next post I will write about what was my first approach before I accidentally fixed the issue.
Hello, world!
This is my very first ever blog post. When I get a little time more will follow!
dfighter1985 