Irclogger

What is irclogger

irclogger is a simple "bot", a program connecting as a client on IRC servers to provide a web log of what is said. It aims to provide a simple, fast, efficient and web-compliant service. It is quite robust and mature, having be in daily heavy use for personal & business use since 2003. This page is at http://colas.nahaboo.net/Software/IrcLogger.

Goals

Irclogger was done to:
  • Be used on intranets, where users can be trusted and will not try constantly to crack the system. Thus the bot do not need to provide the plethora of anti-hackers, channel defending measures
  • Provide privacy, users can set passwords on the log files, change them, but cannot remove them
  • Be another web tool, the bot obeys just the minimal IRC commands to log or not a channel. All the rest is done via a web browser. Each logged phrase becomes a part of the web with its own URL.

Screenshots

Click on an image to enlarge
logs_list.jpg The list of logs, latest first, weekdays are bigger, a space separates weeks, number of lines given to get a sense of importance, available in many formats
log.jpg A day log, Users names have color names spread in the spectrum, rows alternate colors, URLs are clickable, idle periods are signalled with aproportional number of dots, admin messages are greyed and smaller, and you can select a line by clicking on the time at the right, and give the URL in the browser bar to other people by email, in web pages, or ... IM, and they will see the same thing as you, that is:
select.jpg Selected line, in the whole page, but positioned on it and highlighted
urls.jpg URL only view, to have a quick summary of all the URLs mentioned in a day log. Click to the time at right to expand on one in context, just like the above view.

User manual

irclogger appears just as another user chatting on the IRC server and channels, with the name [LOGGER]. You use it by talking to it, just like another user, with the /msg command on most clients, or open a private discussion from the menus.
  • To start logging a channel "#foo", either:
    • invite the bot to the channel. In channel #foo, type:
      /invite [LOGGER]
      Depending on the server configuration, you may not have the rights to do this, so just use the second solution:
    • tell it to log the channel, by sending it a private messages "log channel", i.e.
      /msg [LOGGER] log #foo
      If the channel is password protected by a key, say "bar", you must append it, e.g:
      /msg [LOGGER] log #foo bar
  • To stop logging a channel, you can use any operation to remove a user, or just say in the channel: [LOGGER] off
  • To type a non-logged line, just prefix it by [off], as in: [off] well, it sucks :-)
  • To get online help, just say anything directly to the bot in a private message

Web logs Each logged line has its own permanent URL, that you can get by clicking on the time. It will be of the form
http://...?date...&sel= line #topline= , e.g: http://my.site?date=2008-09-17&sel 26 #22=
(topline = line -4). You can select a range of lines to refer to from other web pages by adding by hand the ending line of the range to the url, as:
http://...?date...&sel=line -lastline #topline= , e.g: http://my.site?date=2008-09-17&sel=27 -43 #23

Privacy

The bot only logs when you can see it in the user list as user " [LOGGER] ". If you do not see it, it cannot log you. The bot do not log newly created channels, it has to be explicitely invited to log them. The bot writes a very visible message in the channel when starting and stopping logging a channel.

Editing logs is easy for an admin as they are only plain text files on the server.

Users can decide to password-protect the web logs. Once protected the log cannot be made unprotected again. Some channels can be declared public by the administrator, which forbids users to password-protect them, by creating file named PUBLIC in their logs directory.

Note: The logs are not protected by default, even if the IRC server or IRC channel themselves are password protected. So be careful to always protect new logs on these servers, or make them avialable through a private web server.

Requirements

  • linux (or an unix with the GNU utilities: grep, sed, ...)
  • python 2.3+
  • a web server. tested only on apache, though.

Installation & update

Detailed installation and configuration info are in the README.txt file in the distribution.

Upgrade is normally just de-tarring the distrib in /usr/local/, and see the HISTORY.txt file in the distribution for optional additional manual upgrade steps. Note that you should uncompress the distribution into a permanent location (I suggest /usr/local/irclogger). The install script will then create an irclogger-run script customized for each instance of the logger you want to run (on instance can log multiple channels, but on a single IRC server), but the actual scripts in the instances will be symbolic links to the installation directory, so that upgrading is just recopying the new files in the install dir.

Implementation

I could not find a bot satisfying my goals. All were huge monsters with bloated functionalities of dubious utility, many were disgusting amounts of ugly perl code.

I stumbled on a nice unfinished python bot " logger" by Sean B. Palmer and Dan Connolly, so I used it as a base for irclogger, and added html view of the logs.

irclogger consists of 2 parts:

  • irclogger, a python IRC client writing text logs in the MIRC format.It takes as options:
  1. the irc server name. e.g irc.freenode.net
  2. the nickname of the bot. A nice, visible [LOGGER] is advised, but can be anything
  3. the directory where the logs will be maintained. No need to be in the web server directory. e.g: /home/irclogger/logs
  4. the urls where the logs can be found. This is just the one displayed in the bot announces and help text in IRC, so you can put the URL of the irclogger_logs script or one page on your site pointing to it with more explanations. e.g: http://my-site.com/irclogger/bin/irclogger_logs
  5. (optional) the channels that will be monitored at start, you can put as many as you want, but be wary that some IRC servers limit drastically the number of channels you can join
  • CGI shell scripts, (bash) to be used from a web server to format the logs into html and provide searching in them.
    • output is available in 4 modes:
      • html (default), with separations when no conversations happened for more than one hour, ans the ability to select a line by clicking on its time at the right margin, which makes the URL a permalink that you can send to other people, they will see the line selected highlighted in context on the page
      • twiki syntax for easy cut and paste to TWiki engine
      • raw just the text, without messages and names of people, useful to compose minutes of meetings
      • urls is the html mode, with only the lines with URLs in them displayed. To restore the full view, either use the toggle in the navigation bar, or click on the time at the right of a line to "zoom in " to it in full view.
    • search is performed by AGrep, which provides an approximate search, very useful for IRC where you often make typos.
    • running irclogger-run is a wrapper script to run the python daemon with the proper options, and restart it if it fails, after a some time. This script is built for you by the installer.

License

Pure Open source: GPL

More details

Importing logs

In the distribution I provide some converters:
  • xchat-to-mirc to convert xchat v1.x logs to the mirc format used by irclogger
  • xchat2-to-mirc to convert xchat v2.x logs to the mirc format used by irclogger

New versions announcements

New releases and important issues will be announced on the Irclogger blog, so I strongly suggest you monitor it, either
  • by RSS to be warned of new releases and major changes (recommended)
  • by email, by email gateways to the above RSS, for instance by
    Subscribe via
  • by email, for instance by Changedetection.com, on:

Sample use

Related Links

History

The full history is in the file HISTORY.txt, but here are the highlights: ALERT! means that there are extra manual steps to be performed on upgrade.
  • 1.16 27 Apr 2010
    • ALERT! MANUAL STEPS for upgrade: you need to restart your irclogger daemon (just kill the irclogger processes, the irclogger-run* launchers will restart them in 45s). Kill the python process not the sheel one, e.g:
      kill $(ps awx|grep '[p]ython.*irclogger'|cut -d ' ' -f 1)
    • BUGFIX: http://trac.colas.nahaboo.net/irclogger/ticket/10 on some IRC servers, a bogus log dir ending in ] was created, making the bot crash.
    • sample /etc/init.d/irclogger startup script for red hat contributed by an user
  • 1.15a 14 Mar 2009
    • warning: now needs a GNU sed (a sed with the -r option for extended regexpes, actually) for html rendering. This is the sed used in Linux.
    • html rendering: now properly exclude dots, commas and ending parenthesis from urls. You can write http://foo/bar, or (http://foo/bar) and it will only make a link to the proper URL. Note that it will still properly match urls with parentheses in them, such as: http://en.wikipedia.org/wiki/Paris_(genus)
    • html rendering now recognizes also urls with https protocol (in addition to http and ftp)
    • bug fix for debug mode (ircloggerd), was missing an import
  • 1.15 17 Feb 2009
    • ALERT! MANUAL STEPS for upgrade: you need to restart your irclogger daemon (just kill the irclogger processes, the irclogger-run* launchers will restart them in 45s)
    • irclogger can now connect to password-protected irc sites. Just set the environment variable IRC_SERVER_PASSWORD in the irclogger-run launcher. No need to upgrade if you connect to password-less servers.
  • 1.14f 01 Feb 2009
    • twiki markup renamed as TML, irclogger_totwiki becomes irclogger_totml
    • bugfix: in search, the "Case sensitive" and ""Verbatim" options results had display bugs (extra newlines). Bug found by Will Norris.
    • MANUAL STEPS for upgrade:
      • ALERT! from install dir: ln -s $PWD/irclogger_cgiparse $BIN
        (hint
        do locate irclogger_log_search to find $BIN dirs)
        E.g: if your install dir is /usr/local/irclogger, and you downloaded the distrib in /tmp:
        1. cd /usr/local
        2. tar xfz /tmp/irclogger-1.14f.tgz
          for BIN in $(locate -r '/irclogger.conf$'); do
          ln -s /usr/local/irclogger/irclogger_totml ${BIN%/*}; done
      • ALERT! you should clear the logs list cache by either calling once your irclogger_logs url with /-cc appended, e.g: http://my-site.foo.com/irclogger/irclogger_logs/-cc or doing a sudo rm -rf  /tmp/.irclogger_logs_cache on the server
        NOTE: this is optional and cosmetic just in order to replace the twiki name by TML.
  • 1.14e 08 Nov 2008 In the logs list, the number of lines now only count the actual lines typed by users, not the control messages (has joined, has quit, ...) lines
    MANUAL STEPS for upgrade:
    • ALERT! you should clear the logs list cache to get the correct line count by either calling once your irclogger_logs url with /-cc appended,
      e.g: wget my-site.foo.com/irclogger/irclogger_logs/-cc
      or doing a sudo rm -rf  /tmp/.irclogger_logs_cache on the server
  • 1.14d 16 Oct 2008 Some clarification on navigation labels
  • 1.14c 15 Oct 2008 ALERT! added a "only urls" mode to only see the lines of the log with URLs in them
  • 1.14b 11 Oct 2008 fix for search result offsetted by one line
  • 1.14a 09 Oct 2008 fix for some agrep versions
  • 1.14 19 Sep 2005 Bugfixes for python 2.3
  • 1.13 10 may 2005 New optional file CHANNELS.deny in logs dir to limit allowable channels
  • 1.12a 15 Feb 2005 ALERT! fix space not recognised in search on some systems
  • 1.12 _ 25 Jan 2005_ [off] at start of lines can now be case-independent
  • 1.11 _ 30 Sep 2004_ Logged channels status were not saved in case of hard machine crash
  • 1.10a 08 Sep 2004 Enhancements to html views
  • 1.10 _ 02 Aug 2004_ ALERT! many fixes, and a cache system for huge speed boost on web view
  • 1.9b 19 Mar 2004 URL shown in the browser is perennial
  • 1.9a 06 Jan 2004 Bugfix in html rendering
  • 1.9 _ 21 Nov 2003_ more tolerance for bot commands
  • 1.8e 15 Nov 2003 dates in URLs can now omit the day of week part:
  • 1.8d 09 Nov 2003 selected lines in the html web rendering can now be a range
  • 1.8c 09 Nov 2003 log files can now be compressed
  • 1.8b 08 Nov 2003 search is now done by agrep
  • 1.8a 06 Nov 2003 small portability fixes
  • 1.8 23 Oct 2003 reconnect to channels on restart
  • 1.7 23 Oct 2003 forces all channel names to lowercase
  • 1.6a 22 Oct 2003 new output format: TWiki
  • 1.6 21 Oct 2003 Minor cosmetic fixes
  • 1.5 20 Oct 2003 published on the web at http://koala.ilog.fr/irclogger entry "irclogger" in freshmeat.net.
  • 1.4 09 Oct 2003 log message put in a single line to avoid flooding
  • 1.3 04 Oct 2003
  • 1.2 13 Sep 2003 should now quit properly on deconnections and not hang indefinitely
  • 1.1 12 Sep 2003 Putting a file PUBLIC in a channel log dir makes it unprotectable
  • 1.0 11 Sep 2003 First fonctional release. Installed on irc.ilog.fr intranet, and on #twiki on overnet

Comments

  • "amette" on #tikiwiki "[irclogger] is a more or less bad webapp, but insanely fast!" (see in context). I agree the code is a big hack, but it is fast and reliable.


RSS feed of this page comments

 
Navigate: Changes -  Index -  Map -  Search -  Print version -  RSS Feed
Advanced: Backlinks -  Children -  Raw View -  Email changes -  History
Admin: Admin -  Statistics -  Preferences -  Notifications -  Your Account
Edit: Edit -  Raw edit -  Attach -  Create New Topic -  More topic actions
Software.IrcLogger
Topic revision: r64 - 2010-05-01 - 19:26:43 - ColasNahaboo
Powered by This site is powered by the TWiki collaboration platformTWiki - Send feedback
Hosted by Linode on Debian