Thoughts on Lucene, Solr, crawling and vertical search 

Introducing Bash-whacking

Posted by Kelvin on 04 Dec 2010 | Tagged as: Ubuntu

Starting a new blog series on bash scripts you shouldn't live without.

Installation for scripts is simple: either add them to /usr/bin or add ~/bin to your bash path and place your scripts there. Don't forget to make the …

Continue reading…

No Comments »

Recursively delete .svn folders

Posted by Kelvin on 04 Dec 2010 | Tagged as: Ubuntu

#!/bin/bash
find ./ -name ".svn" | xargs rm -Rf
 

2 Comments »

Move up multiple directories in bash

Posted by Kelvin on 04 Dec 2010 | Tagged as: Ubuntu

Ever thought typing cd.. 5 times was silly? Try this up() function!

Add this to .bash_profile or .bashrc

up () {
    if [ -z $1 ]; then

…

Continue reading…

No Comments »

Download youtube videos as mp3

Posted by Kelvin on 04 Dec 2010 | Tagged as: Ubuntu

For Ubuntu..

sudo apt-get install ffmpeg

cd ~/bin
wget https://github.com/rg3/youtube-dl/raw/2010.11.19/youtube-dl
chmod +x youtube-dl

wget http://www.supermind.org/scripts/youtube2mp3
chmod +x youtube2mp3

 

The contents of youtube2mp3 is simple:

Continue reading…

No Comments »

Useful ffmpeg commands

Posted by Kelvin on 06 Dec 2010 | Tagged as: Ubuntu

Shameless ripped from http://www.catswhocode.com/blog/19-ffmpeg-commands-for-all-needs

Getting infos from a video file

ffmpeg -i video.avi

Turn X images to a video sequence

ffmpeg -f image2 -i image%d.jpg video.mpg
This command will transform all the images from the current directory …

Continue reading…

No Comments »

Recursively copy only files with certain extension with rsync

Posted by Kelvin on 16 Jan 2011 | Tagged as: Ubuntu

Sometimes you want to get to all MP3 or PDF files which are nested in a deep directory tree. Here's what you do..

rsync -rvtW –delay-updates –modify-window=1 –progress –include='*.pdf' –include='*.xml' –exclude='*.*' source dest
 

No Comments »

Recursive directory listing sorted by file size

Posted by Kelvin on 03 Dec 2010 | Tagged as: Ubuntu

Ever wanted to list a directory sorted by decreasing file size? Useful for finding large files..

du -k * | sort -nr | cut -f2 | xargs -d '\n' du -sh
 

Courtesy of LinuxQuestions.org

No Comments »

Download google video and youtube from command-line

Posted by Kelvin on 22 Feb 2011 | Tagged as: Ubuntu

clive is an awesome tool for downloading google video and youtube flash videos from the command-line.

sudo apt-get install clive

# download google video in flv
clive http://video.google.com/videoplay?docid=-1388254997387483895

# download youtube

…

Continue reading…

1 Comment »

Reset wireless in Ubuntu 10.04 Lucid

Posted by Kelvin on 04 Mar 2011 | Tagged as: Ubuntu

Ever had your Intel wireless card play up on you in Ubuntu? You can't seem to reset it like the ethernet cards (i.e. with ifup and ifdown). Do this instead:

sudo rmmod iwlagn &&

…

Continue reading…

No Comments »

Delete files older than x days

Posted by Kelvin on 22 Mar 2011 | Tagged as: Ubuntu

Delete files older than 7 days:

cd /path/to/dir
find -mtime +7 -exec rm {} \;
 

Change +7 to +14 for deleting files older than 14 days.

No Comments »

Next Page »
05/19/2012 | Kelvin Tan | Lucene Solr Crawl Consultant