Feeds:
Posts
Comments

Archive for the ‘Software Development’ Category

Contrary to most pure hardware tools like a hammer, software tends to evolve over time. These days, software evolves faster than ever before – and at the same time most pieces of software that we use regularly are also interconnected with other software. Think of your smartphone, where the operating system updates the apps running on the device, while some – if not most – of the apps require connections to other infrastructural software and “platforms” from the likes of Google, Apple, and many others. Synchronising account and application data is getting more important every day, the more so now that more and more people have more than one device. No wonder then than sometimes things take a turn for the worst…

Case number one: I have been using a couple of home-brewed scripts to get the daily production numbers of our solar panels from the SMA monitor to an Xubuntu computer, and then transfer them to a Google Drive for storage. I used Grive2 to sync new or renewed files to Google Drive, until that failed as I reported on December 15th. Google started restricting OAuth access rights in November 2019, and that poses a problem for tools like Grive2.

My replacement solution using Jdrivesync is actually victim of the same OAuth change, although it is less evident: it can still add files to Drive but fails when reading the metadata of Drive files (and hence is incapable of replacing them as well).

Today I took the time to tackle the issue head-on, and started by re-reading the instructions on Grive2. That answered my question of a few months ago: I now know why Google changed its approach. The Grive2 site also explains how to circumvent the limitations, by creating your own Google API project and OAuth credentials. It’s not the fault of the Grive2 author, but man oh man, what a convoluted process is that. You get to answer a pleiad of questions that may be easy to understand for a seasoned Google developer, but not for an end user trying to get a simple sync script to work again! In the end, after a series of dire warnings by Google during the process, things started working again. Which is nice. But I’m still not sure for how long this will continue to work. That’s not reassuring for a solution that is supposed to work without a hitch for at least 10 more years or so.

I think the burden here is on Google: it would be nice if they could figure out a way for single end users to get a single application instance (project) up and running on a single account in an understandable process. Because that is what I needed: a way to tell Google that MY Grive2 script will sync MY data from MY computer to MY Google Drive. A simple process does not need to bother me with questions about GSuite domains, privacy declarations, consent screens, and what more. Please, Google?

Case number two: since a few weeks I’m a happy user of KeePassium. I use it on my iPhone as well as on an iPad, where both devices open the same KDBX file. Since I also still have an Android device running Keepass2Android, I store the KDBX file in DropBox. This setup seemed to work OK, until a few days ago when a new account added on the iPad did NOT show up on the iPhone nor in Keepass2Android. After a few tests and trials I ended up with saving the file explicitly to DropBox and reopening it on both the iOS devices, and later synced Keepass2Android as well. The latest changes in the file are now visible on all three machines, so that’s good.

However, I fear that I may have lost one earlier password change. I’m not in any position to blame either DropBox, Apple’s Files app, or KeePassium, since I cannot (yet?) explain what happened. So while the situation is “under (manual) control” now, I keep wondering what will happen when I apply the next changes to the KBDX file. Here, like in the case above, the synchronisation should ideally happen without any special interaction on my part. Unfortunately, as long as I’m not certain that the complete setup works “as expected” I may as well continue to sync by hand – and that is exactly what smart software is supposed to automate, no?

Conclusion? As a developer of sorts, I’m familiar with all aspects of software, good and bad alike. I know things can go awry, and I know how to try and figure out what goes wrong and how to try and resolve the issue. But I’m part of a minority, speaking globally, and I can imagine that many (most) people would just declare defeat and call the software they were using “buggy” or “bad” or “useless”. While that may true in some cases, it mostly shows that developers and publishers of software will need to take more care when building their products: no software is an island, and many if not all software tools will have to talk to others – hopefully in a polite and productive manner. Not an easy task, but possibly essential if the tool has to be around for a long time.

Read Full Post »

“Seeing” things as colors or sounds has always intrigued me, so I had to have a look at the “What Color Is Your Name?” website. Don’t expect an extensive and scientific explanation of the phenomenon; just enjoy the results. Here’s what the alphabet look s like for Bernadette:

I can see this site being used to select a color scheme by website designers!

Read Full Post »

A good month ago, I had to switch from Grive2 to Jdrivesync on my little Xubuntu machine, because Google doesn’t like the former software. Unfortunately, Jdrivesync is not without problems.

The biggest issue is that Jdrivesync is not capable of updating an existing file in de Google Drive with a fresher version from my machine. And it turns out that I’m not the only one (nor the first) one to experience this error, as detailed in this Github error report called “Error if updating a remote file“.

I’m the first one to admit that software without bugs is very, very, very rare ;-)

But a bug report without response in more than 20 months is a clear sign of abandoned software. So I’m looking for another solution – suggestions are more than welcome (I’m not in a position to start learning the ins and outs of the Drive API to see if I can find the cause of the problem).

Read Full Post »

In “The State of the Octoverse ” for 2019 Github writes:

And for the first time, Python outranked Java as the second most popular language on GitHub by repository contributors.

Not bad – JavaScript takes first place, Python and Java follow.

Talking about Python allows me to announce that I hacked my first Python application ;-)

You see, I’m still running an old version of Trac to track issues and document my developments. Old means: version 0.10.5, and the problem is that I don’t have the time at this moment to upgrade to the latest version. So when I had to move to a brand-new Windows 2016 server I just took the existing installation instructions and got Trac running fine, although not under the wings of IIS. Except for one thing: the new server was behind a firewall and a virtual IP address (VIP), the latter also taking care of HTTPS to HTTP conversion. So when my browser sends an HTTPS request, Trac sees HTTP traffic and responds with HTTP redirects when replying to certain actions. Those replies turn into error pages in the browser, since HTTP traffic from the browser does not pass the firewall and is not intercepted by the VIP…

I have been reading up on Python, but I can’t say I have developed anything seriously in it. Python is however sufficiently readable so that I easily found my way in the code. To solve my problem, I just had to force Trac to always redirect to HTTPS URLs. As far as I can tell, I had to change but a single line of code to make that happen. Here’s the code of the ‘_reconstruct_url’ function after my hack:

Image of the Python code for the function '_reconstruct_url'

I changed lines 226-227 to the single statement in line 228

I’m calling this a hack because it only solves my problem; this is not something you want to submit to a public repo. It also means that the application no longer responds to standard HTTP requests. But it’s good enough for now.

From what I have seen on the Trac website there are solutions for running later versions of Trac over SSL, but most (all?) of them seem to rely on an SSL front on the same server as Trac. Too bad that such a solution won’t work for me. Guess I’ll have to study Python and Trac a lot deeper to find the best way to cope with the architecture at work.

Read Full Post »

So you’re developing a ColdFusion application and you need to access the output of a Windows program, like ‘netstat.exe‘ to take the example from the Adobe documentation?

The documentation on how to do that is relatively complete, but fails to emphasise an important point. Try this, and you’ll see… nothing, no output:

<cfexecute name="C:/Windows..../netstat.exe" variable="output" />
<cfdump var="#output#" />

The important point is that you really need to specify a non-zero ‘timeout‘ argument if want to capture the output of the Windows tool in a variable. Simple, but yes: this one had me stumped for 90 seconds!

<cfexecute name="C:/Windows..../netstat.exe" variable="output" timeout="1" >
<cfdump var="#output#" />

If you forget that ‘timeout‘ argument the command you issue will execute, but you won’t know its output, since ColdFusion is already further along in its execution. You may have to experiment a bit with the value for the timeout, depending on the specifics of the machine you’re running ColdFusion on and the execution time of the command called (obviously). Not specifying the timeout will launch the command, but it won’t wait for the command to end – which can be useful as well – but not always.

Read Full Post »

Since many years I have been writing down the guidelines I use when developing software. Mainly because it’s easy to forget some of them, but also because at times you need to explicitly ponder their weight in a trade-off between two or more “principles”. One thing is clear after all those years I have spent writing and reading software, whether it was written by me or someone else: writing good software and writing good code is difficult.

Michael Foord has written up his “30 best practices for software development and testing“, and it’s hard to disagree with his list: there is a lot of good advice in it! If you’re just starting to code, you can’t hope to apply all 30 “rules” at once. The only gripe I have is with the title of his post: in my view, it should have been published as “30 best practices of software coding and testing“. There is, after all, a lot more to software development than coding.

Let me prove that statement. For heaps of excellent advice on the subject of software development in general I always pick up Alan M. Davis’s “201 principles of software development“.

My copy was printed in 1995, and it seems there are no reprints – which is a shame, because Alan Davis did a great job of consolidating proven principles in categories ranging from requirements analysis over design to coding and testing and product assurance. Each principle is explained, and Davis adds at least one reference text for almost every principle. In summary, the content of this book is not very original but it remains a valid and comprehensive overview of good software engineering practices. That remains true, even after more than 20 years!

PS. I already wrote earlier posts on the subject of software engineering principles, e.g. in May 2015

Read Full Post »

I encountered this weird situation last week: while testing ColdFusion code from a previous decade, I would see the output of part of the screen twice. Just like that – I had not changed a thing to the logic of the code, just cleaned up a bit of what I consider to be “bad” formatting in code written by others a long, long time ago (2005 or so).

I spent hours of debugging the code, even invited a few colleagues to help me – and then suddenly it struck me: CFML may look like XML, but it isn’t XML. Some CFML tags do their thing a second time when the tag is explicitly closed. One of those tag is <cfmodule>. I never use it myself – I prefer components and functions. But the code under scrutiny included a <cfmodule template='whatever.cfm >, and to “clean things up” I had added a slash at the end, like this: <cfmodule template='whatever.cfm />. Hence the double execution. So don’t do that, will you? Avoid closing <cfmodule>!

You can imagine my state of mind when I realised my error, don’t you? Yeah… I had seen this problem before, but much too long ago to actively avoid it in practice.

Before lecturing me on the subject: I know, I should be writing cfscript… but I don’t have the luxury to rewrite code that is more than 10 years old. So it will stay as it is.

Also: I consider this to be a bug in Adobe ColdFusion (at least in versions 6, 8 and 11). There’s no logical reason for such behaviour. I don’t know if Lucee or BlueDragon act similarly, but I hope they’re smarter than that!

And for the record: for me, code is “badly” formatted when it is hard to read – by me. I know compilers and computers have no trouble handling things like whole programs on a single line, or lacking coherent indents (unless it’s Python, of course), or unclosed XML tags (elements), to name but a few examples. I want code to easily readable by humans, and certainly by me – especially if I’m responsible for its maintenance…

Read Full Post »

Older Posts »