Video surveillance software isn't the lightest thing in the world. On a headless server like Linux without a GUI, it's very likely that it'll be one of the top processes. Can you log in to the system? At the DOS style prompt you'll have to type the username, hit enter, followed by the password. Note that the password at this prompt will be entirely blank. I'm pretty certain you won't even see **** characters showing up to denote each keystroke.
If you can log in, just type "top" and hit enter. This will bring up a command line system manager. It'll show your running processes. If you take a look at some of the ones listed at the top of the list you may get a clue. You can hit CTRL C to cancel out of the top menu.
Some Linux video surveillance packages that come to mind first is Motion, ZoneMinder, Bluecherry, and Xeoma. I have minimal experience with Xeoma as I didn't care for the interface, but I've used the other three.
Motion is terminal-based only. It's a daemon that runs in the background. It has no interface. The basic idea here is Motion is set up to simply record events to a dedicated directory (folder) for that camera. When I used Motion I set it up to record the front camera to front_camera_outside folder, etc etc. Given there's no interface, there's not really a way to interface directly with the cameras like you see some systems bundled with. Motion is a super light, yet quick and dirty way to rig up an old box to house events. If it's using Motion you'll likely see /usr/bin/motion in the list from top.
ZoneMinder works in a similar way (daemon in the background), but it has a web interface to work with. It's web based. If you find out the IP of the Linux box, you can try to hit
http://ip.of.zoneminder.box/zm in a web browser. If you get a web interface with a login prompt, then there you have it. ZoneMinder is pretty rough around the edges, but it does sorta-kinda-sometimes do its job.
Bluecherry is another one. Bluecherry has a server + client relationship with its software. The server runs bluecherry server, and your client machine (laptop, desktop, etc) run the client software. The client software is available on Mac, Linux, and Windows. Bluecherry server runs in the background as a daemon and does not need an interface on the server whatsoever. The interface comes via the client. If you find Bluecherry is what's running on this system, you can simply install the client on your laptop or desktop, punch in the IP and login info to Bluecherry (if you know it), and you'll be in business.
Linux based video surveillance is something I have a bit of interest in. Post back here and I'll help best I can. The above is just some rough ideas that came to mind regarding what I suspect may be running on that Linux box of yours. Something tells me, if anything, it'll be ZoneMinder, but that's strictly a guess. The real gold is going to be what you find when you run the top command.