WebSVN - hotsanic - Blame - Rev 36 - /trunk/Documentation/README

1

simandl

HotSaNIC

2

HTML overview to System and Network Information Center

3

by Bernd Pissny ( hotsanic@bernisys.prima.de / http://www.bernisys.prima.de )

4

5

----------------------------------------------------------------------

6

7

HotSaNIC documentation

CONTENTS

-------------------------------------------------------------------

1. description

2. requirements

3. configuration

4. starting & stopping

5. web output

6. troubleshooting

7. upgrading

8. tools

9. contacting me

10. credits

11. future plans

DESCRIPTION

-------------------------------------------------------------------

28

29

HotSaNIC (the acronym stands for "html overview to system and network

30

information center") consists of a set of perl-scripts built on Tobias

31

Oetiker's "rrdtool" to generate graphical system-statistics, currently

32

supported platforms are linux (main development) and *BSD.

33

I started to build this tool in december 2000, because none of the

34

existing stats-tools gave me the flexibility and resolution i expected.

35

Most tools had a minimum query-time of one minute, while i use a 10sec.

36

timebase, which shows a lot more dynamics in the graphs.

37

The whole project is built in a modular way to make it quite easy to

38

expand, unused modules may be switched off easily.

39

Another advantage over most existing tools is the min/max-area that is

40

drawn behind the average-graphs in many modules - a feature that

41

reflects e.g. min/max bandwidth usage in much more detail than similar

42

tools do.

43

The smallest time span that will be displayed covers the last hour, the

44

longest span diagram covers the last year though all data will be kept

45

for about two years (in order to have the chance to look further back

46

in time and compare). Currently supported data-sources are traffic

47

(local and via SNMP), system's properties like

48

processes/memory/users/loadavg, hdd usage, hdd throughput, ping-times,

49

lm_sensors, #of running copies of given applications, distributed.net

50

statistics, worm-impacts (by analyzing apache-logfiles), APC-USV

51

statistics (load/temperature/charge...), connections to shoutcast

52

streams. More modules are to come on users' requests and ideas.

53

I recently started adding some tools for debugging and to generate

54

alarm-reports (for example if a given threshold is exceeded) which may

55

be mailed to the system's admins.

REQUIREMENTS

-------------------------------------------------------------------

60

61

Tobias Oetiker's "rrdtool" is needed to let this script work. It is

62

used to manage the databases and generate the graphics. At this point i

63

would like to congratulate Tobi, he did a very good job writing this

64

magnificent tool!

65

66

Since I mainly use Perl5 to sample the data and update the databases,

67

of course this has to be installed on your system. But this should not

68

be an issue since most systems already come with full Perl5 support.

69

But please be sure to install the perl-libraries of "rrdtool" using

70

"make site-perl-install" afer you install "rrdtool."

71

72

"convert" from the "ImageMagick" package has to be installed to get

73

full functionality.

74

75

"iptables" or "ipchains" has to be installed if you want to use the

76

"network" module. Maybe you have to build a new kernel which has the

77

corrosponding firewalling capabilities (packet filter)

CONFIGURATION

-------------------------------------------------------------------

82

83

For version 0.3.6 first of all you should run setup.pl to generate the

84

start/stop script (this will be rrdgraph) and the main "settings" file.

85

The setup-script also calls each module's setup script to generate all

86

module-specific settings. After that you should check if the "settings"

87

file in the main directory contains the correct configuration to fit

88

your system. Then you should check the settings files in each module

89

directory and edit the generated "settings" files to fit your system.

90

91

The "settings" files are (well, ok, at least they should be) quite

92

self-explaining, so i guess it isn't needed to explain them here -

93

though some day I might list every single module on this page along

94

with a short explanation of what they do and which features they have.

95

96

STARTING AND STOPPING THE DAEMON

97

98

-------------------------------------------------------------------

99

100

To run the auto-update daemon the best thing would be to get yourself

101

root-permissions and enter:

./rrdgraph start

rrdgraph will begin sampling data vor every plugin installed every 10

106

seconds.

107

108

To stop the daemon enter

./rrdgraph stop

Most modules will run as non-root, too, but if you want full

113

functionality it is absolutely necessary to let the whole thing run as

114

root. For example the PING module uses ICMP stuff that is only working

115

if you are running it as root.

116

117

GENERATING WEB-STATISTICS

118

119

-------------------------------------------------------------------

120

121

Well, you should not have to worry about anything except the

122

output-directory which has to be configured correctly (you will have to

123

enter the full path!) in the main settings file.

124

125

The images on the statistics pages will be built automatically by

126

HotSaNIC every configured timespan (which will be 15min. by default) If

127

you like to build the graphs more often, you are welcome to decrease

128

the time-base as you like by editing the main settings file. But be

129

sure that decreasing the time-span will eat up more CPU time (which may

130

be needed for more important tasks ... ).

131

132

To build the index.html files just start makeindex.pl and all shall be

133

well. The script will gather every plugin-directory and build the

134

necessary index-files for your webpage. In case you install a new

135

plugin, just let HotSaNIC run a couple of minutes inititalize the

136

plugin's databases. Then you just call makeindex.pl again and it will

137

do the rather nasty update-job for you. New plugins can be installed on

138

the fly, the only thing to do is to copy the module-directory to the

139

main directory and maybe you have to add the new module to the RUN and

140

SHOW entries in the main settings.

141

142

If you use the "traffic" plugin, please let rrdgraph run at least some

143

30 seconds to allow the plugin to scan your local interfaces before you

144

run the index-maker.

145

146

"rrdtimer" will call "convert.sh" about every 24h (timebase may be

147

configured in main settings file). This script will generate pictograms

148

on the main index page of your webstats from the actual weekly graphs.

149

150

This will require "convert" from "ImageMagick" to be installed. It is

151

used to resize the weekly graph images.

TROUBLESHOOTING

-------------------------------------------------------------------

156

157

o How do I solve the "cannot find "RRDs.pm" issue?

158

159

-------------------------------------------------------------------

160

161

You probably forgot to install the site-perl module of rrdtool.

162

163

If you installed rrdtool from the source files, you just have to do

164

a

165

166

make site-perl-install

167

168

If you used a .rpm package, you should check if the RRDs.pm has

169

been installed in the correct places. maybe you used an RPM package

170

which didn't really fit your installation - some pathes may have

171

changed.

172

173

Or maybe you upgraded PERL, then you also may have to adapt the

174

modules' pathes.

175

176

o ping module is not working

177

178

-------------------------------------------------------------------

179

180

On some systems the PERL header-files may be missing. This results

181

in the ping-module complaining about something like

182

183

"Can't locate sys/syscall.ph in @INC (did you run h2ph?)"

184

185

Linux distributions known to be affected are for example "RedHat

186

7.2" and "Slackware"

187

188

The solution would be to do (as root) the following:

189

190

cd /usr/include

191

find . -name '*.h' -print | xargs h2ph

192

193

After that everything should run as expected. - hopefully ;)

194

195

o the graphs don't show up

196

197

-------------------------------------------------------------------

198

199

Maybe you just started the daemon. in that case let it run for some

200

minutes to let the databases be created and filled. Then you may

201

run the "diagrams" script in the main directory to generate the

202

graphs. This is done by the daemon automatically every DTIME (found

203

in the main settings)

204

If just the small graphs on the main page don't show, you have to

205

run "convert.pl" (or "convert.sh") in order to create the thumbnail

206

images. This is done every CTIME.

207

208

o the thumbnails on the main page are not updating or still don't

209

exist

210

211

-------------------------------------------------------------------

212

213

Maybe you upgraded to v0.3.6 or later ?

214

In the later snapshots of this version the thumbnails moved from

215

the main web-dir to the module subdirs to enhance the overview a

216

bit. You have to call the main "makeindex.pl" to update the html

217

pages once again.

218

219

o after i compiled the scripts some modules fill the logfiles with

220

errors

221

222

-------------------------------------------------------------------

223

224

At the moment there is no other way than removing the binaries. I

225

had such strange effects on some systems myself, but I don't know

226

why the binaries behave that way. The compiling completes without

227

any errors, but the binaries won't start properly like the .pl

scripts do.

UPGRADING

-------------------------------------------------------------------

233

234

Generally the upgrade process involves the following actions:

235

o stop the daemon

236

o copy the new files over the existing ones

237

o call the main setup script

238

o call each module's update script (if exists)

239

o re-start the daemon

240

However, if you intend to upgrade your HotSaNIC installation, you

241

should check the HotaNIC homepage firt to make sure wich additional

242

steps have to be performed. Maybe you have to check the history as

243

well, in case you update from a very old version.

244

Usually updates run quite smoothly without any further changes. You

245

should do as described above: Stop the daemon first to make sure

246

that no data will be fed into the databases which may lead to some

247

annoying side-effects. Then copy the new files to the corrosponding

248

directories. If some major changes occur, there will be an update

249

script in the affected module which you will have to call. The

250

script will take care the necessary and sometimes really nasty

251

modifications for you.

252

Anyway, if you copy a newer version over a previous one, it's

253

always a good idea to call the main setup script once again. Maybe

254

some new important configuration items have been added since you

255

last upgraded or installed the tool!

256

Oh, and don't forget to re-start the daemon afterwards ;).

TOOLS

-------------------------------------------------------------------

261

262

Since version 0.4.0 I added a directory containing some possibly

263

useful scripts and add-ons for HotSaNIC.

264

A brief description resides in this directory, some further

265

descriptions may appear in this documentation later.

266

267

One nifty thing has to be mentioned:

268

The modules may be compied in order to start up a bit faster. This

269

will save some percent of your valuable CPU time, but it may not

270

run on every machine! Unhappily the behaviour of the binaries is

271

beyond my control. In most cases they run as smoothly as the

272

perl-scripts, but up to three times faster, sometimes they fill the

273

error-log and sometimes they won't even compile. Only thing you can

274

do is to give it a try and remove all binaries which cause

275

problems.

276

277

The compilation process is quite easy: Just call the "compile"

278

script in the main directory - that's all.

279

A whole load of compiler information will scroll down your screen

280

for quite a while. During the (hopefully successful) compiling

281

process, your daemon will automatically start to call the binaries

282

instead of the scripts. These are named "read-data" (without the

283

.pl suffix). This allows to compile while the daemon is already

284

running.

285

The compile script has some useful options:

286

o "compile r" forces the re-compiling of all modules - no matter

287

if they are already compiled or not.

288

o "compile u" removes all binaries, leaving just the scripts

289

again.

290

o "compile c" checks if any modules have to be re-compiled (does

291

not change anything, just lists them)

292

o "compile " compiles precisely not more the given module (the

293

prefix "data-" may be omitted).

CONTACT

-------------------------------------------------------------------

298

299

If you encounter further problems or have any questions or new

300

ideas, please contact us at hotsanic@bernisys.prima.de, we will try

301

to take care of everything. It would be a good idea to put the

302

keyword "hotsanic" and a brief description of your request into the

303

subject to make sure i find your mails again in case we need them

304

later on.

305

306

If you already have a solution for a problem you encountered,

307

please attach the working code or the modifications you made. A

308

proper description of your problem and the changes you made would

309

be very helpful.

310

311

Same goes for extensions ideas and support for systems other than

Linux.

CREDITS

-------------------------------------------------------------------

317

318

At this point i'd like to thank and honor some people who

319

support(ed) me and my project:

320

o Matt Burke and Peter Reich, who are heavily involved in

321

getting all functions running on BSD systems !

322

o Todd Underwood, who managed to create some RPMs !

323

o All my trusted friends for being appreciative of my hacking

324

till late in the night ! ;)

325

o Everybody who sent me hints, new ideas, extensions and

bug-reports.

FUTURE PLANS

-------------------------------------------------------------------

331

332

I intend to convert the whole shell-script crap to a PERL script

333

which will run very much faster and therefore does not generate

334

unnecessary system-overhead. This is already done for the

335

data-sampling scripts and the index generation. TODO: diagrams

336

337

I would like to implement more SMNP features not only for the

338

traffic module (as far as possible of course).

339

340

In case you have written your own plug-in and you find it useful,

341

be sure to e-mail me a .tgz of your module and include a short

342

description. Same goes for reporting bugs of course!

Thank you!

----------------------------------------------------------------------