Contents:
Irclogger
What is irclogger
irclogger is a simple "bot", a program connecting as a client on IRC servers to provide a web log of what is said. It aims to provide a simple, fast, efficient and web-compliant service. It is quite robust and mature, having be in daily heavy use for personal & business use since 2003. This page is at https://colas.nahaboo.net/Code/IrcLogger.Goals
Irclogger was done to:- Be used on intranets, where users can be trusted and will not try constantly to crack the system. Thus the bot do not need to provide the plethora of anti-hackers, channel defending measures
- Provide privacy, users can set passwords on the log files, change them, but cannot remove them
- Be another web tool, the bot obeys just the minimal IRC commands to log or not a channel. All the rest is done via a web browser. Each logged phrase becomes a part of the web with its own URL.
Screenshots
Click on an image to enlarge | | The list of logs, latest first, weekdays are bigger, a space separates weeks, number of lines given to get a sense of importance, available in many formats | | A day log, Users names have color names spread in the spectrum, rows alternate colors, URLs are clickable, idle periods are signalled with aproportional number of dots, admin messages are greyed and smaller, and you can select a line by clicking on the time at the right, and give the URL in the browser bar to other people by email, in web pages, or ... IM, and they will see the same thing as you, that is: | | Selected line, in the whole page, but positioned on it and highlighted | | URL only view, to have a quick summary of all the URLs mentioned in a day log. Click to the time at right to expand on one in context, just like the above view. |User manual
irclogger appears just as another user chatting on the IRC server and channels, with the name [LOGGER]. You use it by talking to it, just like another user, with the /msg command on most clients, or open a private discussion from the menus.- To start logging a channel "#foo", either:
- invite the bot to the channel. In channel #foo, type:
/invite [LOGGER]
Depending on the server configuration, you may not have the rights to do this, so just use the second solution: - tell it to log the channel, by sending it a private messages "log channel", i.e.
/msg [LOGGER] log #foo
If the channel is password protected by a key, say "bar", you must append it, e.g:
/msg [LOGGER] log #foo bar
- invite the bot to the channel. In channel #foo, type:
- To stop logging a channel, you can use any operation to remove a user, or just say in the channel:
[LOGGER] off
- To type a non-logged line, just prefix it by [off], as in:
[off] well, it sucks :-)
- To get online help, just say anything directly to the bot in a private message
http://...?date
...&sel= line #topline= , e.g: http://my.site?date=2008-09-17&sel
26 #22= (topline = line -4). You can select a range of lines to refer to from other web pages by adding by hand the ending line of the range to the url, as:
http://...?date
...&sel=line -lastline #topline= , e.g: http://my.site?date=2008-09-17&sel=27 -43 #23
Privacy
The bot only logs when you can see it in the user list as user " [LOGGER] ". If you do not see it, it cannot log you. The bot do not log newly created channels, it has to be explicitely invited to log them. The bot writes a very visible message in the channel when starting and stopping logging a channel. Editing logs is easy for an admin as they are only plain text files on the server. Users can decide to password-protect the web logs. Once protected the log cannot be made unprotected again. Some channels can be declared public by the administrator, which forbids users to password-protect them, by creating file namedPUBLIC
in their logs directory.
Note: The logs are not protected by default, even if the IRC server or IRC channel themselves are password protected. So be careful to always protect new logs on these servers, or make them avialable through a private web server.
Requirements
- linux (or an unix with the GNU utilities: grep, sed, ...)
- python 2.3+
- a web server. tested only on apache, though.
Installation & update
Detailed installation and configuration info are in the README.txt file in the distribution. Upgrade is normally just de-tarring the distrib in/usr/local/
,
and see the HISTORY.txt file in the distribution for optional additional manual upgrade steps. Note that you should uncompress the distribution into a permanent location (I suggest /usr/local/irclogger
). The install
script will then create an irclogger-run
script customized for each instance of the logger you want to run (on instance can log multiple channels, but on a single IRC server), but the actual scripts in the instances will be symbolic links to the installation directory, so that upgrading is just recopying the new files in the install dir.
Customization
Customization is done via a fileirclogger.conf
in the installation directory. It is a file in bash syntax that can define configuration variables, eher with their default value: -
logsdir=/var/log/irclogger
Where are the logged data stored -
passfile=/var/www/passwds/irclogger-passwds
Where are channels password stored -
private_image=IMG
The HTML IMG tag displaying the "private" indicator next to a channel name. Default is a key. -
top=URL
where the upmost "back" link will bring the user -
search_timeout=N
where N is a timeout in seconds to limit the time a search takes. Defaults to 60 -
cachedir=path
where the list of logs will be cached. Default to /tmp/.irclogger_logs_cache -
cachepages=on
if non empty, will also cache the individual pages into cachedir above (default is not to cache)
$logsdir
, one can put files: -
style.css
that will be included in all html pages -
message.html
will be included at the top of the html index page
Implementation
I could not find a bot satisfying my goals. All were huge monsters with bloated functionalities of dubious utility, many were disgusting amounts of ugly perl code. I stumbled on a nice unfinished python bot " logger" by Sean B. Palmer and Dan Connolly, so I used it as a base for irclogger, and added html view of the logs. irclogger consists of 2 parts:- irclogger, a python IRC client writing text logs in the MIRC format.It takes as options:
- the irc server name. e.g irc.freenode.net
- the nickname of the bot. A nice, visible
[LOGGER]
is advised, but can be anything - the directory where the logs will be maintained. No need to be in the web server directory. e.g:
/home/irclogger/logs
- the urls where the logs can be found. This is just the one displayed in the bot announces and help text in IRC, so you can put the URL of the
irclogger_logs
script or one page on your site pointing to it with more explanations. e.g:http://my-site.com/irclogger/bin/irclogger_logs
- (optional) the channels that will be monitored at start, you can put as many as you want, but be wary that some IRC servers limit drastically the number of channels you can join
- CGI shell scripts, (bash) to be used from a web server to format the logs into html and provide searching in them.
- output is available in 4 modes:
- html (default), with separations when no conversations happened for more than one hour, ans the ability to select a line by clicking on its time at the right margin, which makes the URL a permalink that you can send to other people, they will see the line selected highlighted in context on the page
- twiki syntax for easy cut and paste to TWiki engine
- raw just the text, without messages and names of people, useful to compose minutes of meetings
- urls is the html mode, with only the lines with URLs in them displayed. To restore the full view, either use the toggle in the navigation bar, or click on the time at the right of a line to "zoom in " to it in full view.
- search is performed by AGrep, which provides an approximate search, very useful for IRC where you often make typos.
- running
irclogger-run
is a wrapper script to run the python daemon with the proper options, and restart it if it fails, after a some time. This script is built for you by the installer.
- output is available in 4 modes:
License
Pure Open source: GPLMore details
Importing logs
In the distribution I provide some converters:-
xchat-to-mirc
to convert xchat v1.x logs to the mirc format used by irclogger -
xchat2-to-mirc
to convert xchat v2.x logs to the mirc format used by irclogger
New versions announcements
New releases and important issues will be announced on the Irclogger blog, so I strongly suggest you monitor it, either- by RSS to be warned of new releases and major changes (recommended)
- by email, by email gateways to the above RSS, for instance by
- by email, for instance by Changedetection.com, on:
- the IrcLoggerLeftBar left column URL (click here to subscribe) which only changes on new stable releases
- this current page (click here to subscribe), but this is not recommended as you will be emailed on each small typo correction
Sample use
- #foswiki logs of the #foswiki
- logs of the #tikiwiki IRC channel
- logs of the #twiki IRC channel
- Wrapper scripts to run irclogger:
- irclogger-run-twikirc: script to run the logger on freenode servers, by rotating through the list of server if we are disconnected, to avoid being stuck on a dead, full, or temperamental server
Related Links
- irclogger has a Freshmeat entry
- François Beerten has started rewriting irclogger in pure python.
History
The full history is in the file https://github.com/ColasNahaboo/irclogger/raw-file/tip/HISTORY.txt HISTORY.txt, but here are the highlights: means that there are extra manual steps to be performed on upgrade.- 1.17 22 Oct 2021
- python scripts (ircAsyncD.py ircAsync.py irclogger ircloggerd) now explicitely require python2 (#!/usr/bin/python2) as "python" alone is phased out of linux distributions.
- 1.16c 16 Jan 2018
- New option cachepages=on to cache individual pages for each day. Default is still only to cache the listings of all pages in a channel. This option may be set in the irclogger.conf file in the installation directory, as for the other options.
- The existing cachedir=path option is now documented. It defines where the cache files are stored, and defaults to /tmp/.irclogger_logs_cache
- No other changes, so no need to upgrade if you do not plan to use the new cachepages=on option
- 1.16b 08 Dec 2014
- The case-sensitive option was bugged. Found & fixed by Christopher Foo
- Searches are now timeouted, that is aborted if not done in 60 seconds. This timeout value is settable via the new variable search_timeout in irclogger.conf as a number in seconds.
- 1.16a 30 Nov 2014
- Fix provided by Christopher Foo, irclogger was confused when users typed a backslash at the end of a line
- You can use space to separate words in the search now, not only ;
- minor fixes
- 1.16 27 Apr 2010
- MANUAL STEPS for upgrade: you need to restart your irclogger daemon (just kill the irclogger processes, the irclogger-run* launchers will restart them in 45s). Kill the python process not the sheel one, e.g:
kill $(ps awx|grep '[p]ython.*irclogger'|cut -d ' ' -f 1)
- BUGFIX: http://trac.colas.nahaboo.net/irclogger/ticket/10 on some IRC servers, a bogus log dir ending in ] was created, making the bot crash.
- sample /etc/init.d/irclogger startup script for red hat contributed by an user
- MANUAL STEPS for upgrade: you need to restart your irclogger daemon (just kill the irclogger processes, the irclogger-run* launchers will restart them in 45s). Kill the python process not the sheel one, e.g:
- 1.15a 14 Mar 2009
- warning: now needs a GNU sed (a sed with the -r option for extended regexpes, actually) for html rendering. This is the sed used in Linux.
- html rendering: now properly exclude dots, commas and ending parenthesis from urls. You can write
http://foo/bar
, or (http://foo/bar
) and it will only make a link to the proper URL. Note that it will still properly match urls with parentheses in them, such as:http://en.wikipedia.org/wiki/Paris_(genus)
- html rendering now recognizes also urls with https protocol (in addition to http and ftp)
- bug fix for debug mode (
ircloggerd
), was missing an import
- 1.15 17 Feb 2009
- MANUAL STEPS for upgrade: you need to restart your irclogger daemon (just kill the irclogger processes, the irclogger-run* launchers will restart them in 45s)
- irclogger can now connect to password-protected irc sites. Just set the environment variable IRC_SERVER_PASSWORD in the irclogger-run launcher. No need to upgrade if you connect to password-less servers.
- 1.14f 01 Feb 2009
- twiki markup renamed as TML, irclogger_totwiki becomes irclogger_totml
- bugfix: in search, the "Case sensitive" and ""Verbatim" options results had display bugs (extra newlines). Bug found by Will Norris.
- MANUAL STEPS for upgrade:
- from install dir: ln -s $PWD/irclogger_cgiparse $BIN
- (hint
- do locate irclogger_log_search to find $BIN dirs)
E.g: if your install dir is/usr/local/irclogger
, and you downloaded the distrib in/tmp
:
-
cd /usr/local
-
tar xfz /tmp/irclogger-1.14f.tgz
for BIN in $(locate -r '/irclogger.conf$'); do
ln -s /usr/local/irclogger/irclogger_totml ${BIN%/*}; done
- you should clear the logs list cache by either calling once your irclogger_logs url with /-cc appended, e.g: http://my-site.foo.com/irclogger/irclogger_logs/-cc or doing a
sudo rm -rf /tmp/.irclogger_logs_cache
on the server
NOTE: this is optional and cosmetic just in order to replace the twiki name by TML.
- from install dir: ln -s $PWD/irclogger_cgiparse $BIN
- 1.14e 08 Nov 2008 In the logs list, the number of lines now only count the actual lines typed by users, not the control messages (has joined, has quit, ...) lines
MANUAL STEPS for upgrade:- you should clear the logs list cache to get the correct line count by either calling once your irclogger_logs url with /-cc appended,
e.g:wget my-site.foo.com/irclogger/irclogger_logs/-cc
or doing asudo rm -rf /tmp/.irclogger_logs_cache
on the server
- you should clear the logs list cache to get the correct line count by either calling once your irclogger_logs url with /-cc appended,
- 1.14d 16 Oct 2008 Some clarification on navigation labels
- 1.14c 15 Oct 2008 added a "only urls" mode to only see the lines of the log with URLs in them
- 1.14b 11 Oct 2008 fix for search result offsetted by one line
- 1.14a 09 Oct 2008 fix for some agrep versions
- 1.14 19 Sep 2005 Bugfixes for python 2.3
- 1.13 10 may 2005 New optional file CHANNELS.deny in logs dir to limit allowable channels
- 1.12a 15 Feb 2005 fix space not recognised in search on some systems
- 1.12 _ 25 Jan 2005_ [off] at start of lines can now be case-independent
- 1.11 _ 30 Sep 2004_ Logged channels status were not saved in case of hard machine crash
- 1.10a 08 Sep 2004 Enhancements to html views
- 1.10 _ 02 Aug 2004_ many fixes, and a cache system for huge speed boost on web view
- 1.9b 19 Mar 2004 URL shown in the browser is perennial
- 1.9a 06 Jan 2004 Bugfix in html rendering
- 1.9 _ 21 Nov 2003_ more tolerance for bot commands
- 1.8e 15 Nov 2003 dates in URLs can now omit the day of week part:
- 1.8d 09 Nov 2003 selected lines in the html web rendering can now be a range
- 1.8c 09 Nov 2003 log files can now be compressed
- 1.8b 08 Nov 2003 search is now done by agrep
- 1.8a 06 Nov 2003 small portability fixes
- 1.8 23 Oct 2003 reconnect to channels on restart
- 1.7 23 Oct 2003 forces all channel names to lowercase
- 1.6a 22 Oct 2003 new output format: TWiki
- 1.6 21 Oct 2003 Minor cosmetic fixes
- 1.5 20 Oct 2003 published on the web at http://koala.ilog.fr/irclogger entry "irclogger" in freshmeat.net.
- 1.4 09 Oct 2003 log message put in a single line to avoid flooding
- 1.3 04 Oct 2003
- 1.2 13 Sep 2003 should now quit properly on deconnections and not hang indefinitely
- 1.1 12 Sep 2003 Putting a file PUBLIC in a channel log dir makes it unprotectable
- 1.0 11 Sep 2003 First fonctional release. Installed on irc.ilog.fr intranet, and on #twiki on overnet
Comments
- "amette" on #tikiwiki "[irclogger] is a more or less bad webapp, but insanely fast!" (see in context). I agree the code is a big hack, but it is fast and reliable.