I may have created some interesting work for myself, or at least an interesting side-project.

One of the tasks I have at my current gig is to write a web interface for users to securely upload data to the company database. Previously this was meant to mimic the current system wherein they use SFTP (FTP over SSH) to upload the files, after which an automated process would pull the files into a set of software that would parse the files and then insert the contents into the database.

Today it was suggested to simply do the parsing (if any) and insert into the database directly from the web app (Servlet) when the file upload is complete. It eliminates the security issue of what to do with the files while they sit on the server waiting to be retreived. Everyone was happy with that suggest, so the project took another turn.

Then I piped in with a suggestion to just replace the FTP server with some software that does the same thing. Heads turned; apparently HTTP is well understood while FTP is still magic. Whatever.

I spoke a little out of turn, spilling that I'd done similar for a previous client wherein the FTP server never actually dealt with the file system, but instead allowed FTP commands to prepare it for retreiving and inserting directly into a database. Raw FTP is little different from the FTP we all think of when we're using an FTP client. I had honestly never heard of "SFTP" or "FTPS" before, but figured how much harder can it be.

It took a while to find it, but I finally did.

SSH is the be-all-end-all of the core. I use SSH all the time; love it! Essentially, and super-wildly understated, an SFTP client makes a connection to the SSH server (sshd) and through its internal mechanism the SSH server forwards the now (optionally) encrypted data to the sftp-server, as configured in the sshd_config file.

Finally, some place I can start. I can simply write an FTP server to replace the sftp-server normally used. Mine will talk SFTP through the SSH connection, and as appropriate interact with the database instead of the filesystem.

Of course, it turns out that the FTP protocol (text and binary on separate ports) is different than the SFTP protocol (all binary on the one connection). I finally (much quicker this time) found the RFC on which the SFTP software is written: SSH File Transfer Protocol.

Now, how to make a Java SFTP server... Doing this with FTP was easy; FTP just opens a port and waits for connections. Connections are made and authenticated (normally via the OS, but in my remake via database data), and authenticated (or if allowed, anonymous) users go about their business. How to handle the "business" was the real tricky part. How to open a port and listen is trivial.

With SFTP it appears that each connection invokes an instance of the sftp-server program to do its interaction. Probably "easy enough" I could take the existing C/C++ code (it's available from OpenSSH) and convert the byte handling to Java. Harder, though, is configuring SSH to invoke something written in Java. It might be easier to write the software in C to just interact with the database instead of the file system, but, heck, I've got half or more of that hard part done...

I started cleaning up my FTP server, and will maybe try to get that published somewhere. If I can hammer this SFTP server down, that'll probably bring me love from a huge community.