What’s with back-save to earlier versions of InDesign?

InDesign can not open files saved by later versions of the application.

This has been a source of a lot of grumbling over the years. I’ve heard all kinds of silly theories as to why this is so — including one that Adobe does this on purpose to “force” you to upgrade. Conspiracy theories are exciting and everything, but there’s really no conspiracies related to InDesign…

This issue will be brought to the forefront once again with the release of InDesign CS5.5. InDesign CS5 will not be able to open InDesign CS5.5 files. Here’s an explanation as to why:

The InDesign file format is in reality one of a database, and InDesign is a (very complex) transactional database application. Every action in InDesign is a database transaction and all these transactions are saved as a database file for InDesign to be able to read. The fact that InDesign works this way allows for many of its advanced features such as unlimited undo and its almost flawless file recovery. (Interestingly enough a lot of “fluff” is saved in the file resulting from previous transactions which get cleared out by doing a “save-as”.)

Anyone who works with databases knows how important it is to get the database structure correct from the start. Once a database has a structure, it’s very hard to change things later on. Any major changes that might be made can be very difficult to implement and inefficient to boot. InDesign suffers from the same issues as standard databases, and if InDesign was locked into a non-changeable database structure, it would make it very difficult to efficiently add some of the new features and make certain application optimizations impossible.

To solve this problem, each version of InDesign has its own database structure, so it can be properly optimized. Newer versions can read the older database format because the format existed when the newer version was created, and the older format is converted to the newer one when the file is opened. Older versions can’t be expected to read a database format which did not exist when the application was released.

But CS5.5 is just a “dot update”.

Yes. Adobe is not marketing CS5.5 as a “full version”, and that’s because it’s targeting a specific market area (digital publishing), but there were some serious work done in InDesign to enable some of the new features which required structural changes.

What’s Adobe doing to make our lives easier?

IDML

Adobe originally create INX (InDesign Exchange format) for back-saving, but it turned out to be less than ideal for its purposes. It was not really human readable, and InDesign 2.0 couldn’t read it at all. It was not really editable, and it was more or less useless for anything other than InDesign (CS and later) reading a document exported as INX up to one version back.

With CS4, Adobe took INX back to the drawing boards and came up with IDML (InDesign Markup Language). IDML (as well as its sister ICML and IDMS formats) is human-readable xml. The properties match those used by scripting, and the files can even be created from scratch. Additionally, Adobe has made every effort to make IDML truly backwards compatible. New features are simply additional XML tags which are ignored by previous versions, and with the release of InDesign CS5.5 it is finally possible to back-save two versions. IDML files saved by CS5.5 can be opened by CS4 with only the “standard” issues that are to be expected when saving back to a previous version with less features than the one which created the file. You can expect there to be text reflow, and new features used will be badly mangled, but the content of the file and basic features should all be preserved.

We have every reason to assume that this pattern will be continued in future versions of InDesign, so despite the fact that CS5.5 files cannot be opened in CS5, we finally have a solution to the back-save problem that people have been complaining about for years.

Long live IDML! :-)

Tags: , , ,

34 Comments

  1. Marco says:

    THANK YOU. I always wondered what was going on with the -save as earlier version’ stuff.

  2. Wolf Eigner says:

    Great explanation, thank you.

  3. Zev says:

    I am sorry, I simply don’t buy it. Third party developers have been able to add features that are much more robust than the CS5.5 features and have still been able to have their documents opened within the same version. Most of the new features are ePub features which (unless Adobe revamped the whole ePub export) was done with scripting. There is no reason they couldn’t have added these features without creating a new version.

  4. Harbs says:

    Hi Zev,

    My info about the necessity to change the format in CS5.5 came from a “little birdie” who I trust very well, and I’m sure knows what he’s talking abou! 😉

    Actually, the whole epub export was redone from scratch and the whole thing is now rewritten in C++. It’s a totally different story than what was there before (and way more robust)!

    I’m not sure what was announced yet, but it goes way beyond epub, and I can envision a number of the new features which would require a change to the database format.

    • Zev says:

      Did they really redo the ePub export? I knew they would regret doing it with scripting. If you look at the SDK you can see they really put a lot of effort into making the ePub export with scripting all for naught. In the end besides all the features that they had to leave out (i.e. Nested styles) because of scripting the performance issue was a huge problem. I wonder how much faster ePub export is in CS5.5.

      Unless they left something out from the InDesign Secrets post it seems ePub is the main new feature. Story cloning seems to be the only thing besides ePub added.

      • Harbs says:

        ePub is big news.

        There’s three main focuses for this release. EPUB, DMP (Digital Magazine Publishing), and accessibilty.

        EPUB was totally redone. It was taken back to the drawing boards and redone from scratch in C++. They really put in an effort to do it right and the results are greatly improved!

        The DMP tools will be integrated into InDesign, and there’s new features (such as linked stories) geared towards the DMP workflow.

        Accessibility in export was greatly improved as well, and the article panel (which very likely resulted in some changes to file structure) and style/tag mapping helps a lot.

  5. John Hawkinson says:

    > Adobe originally create INX (InDesign Exchange format) for back-saving, but it turned out to be less than ideal for its purposes.

    I still don’t buy this. Maybe INX was marketed that way, but I don’t think that’s how it was engineered. It seems pretty clear it was engineered as a way for 3rd party tools to get data out of InDesign without having to either use the plugin architecture (really not suited for this!) or learn to parse INDD (not well documented and not a stable format).

    Anyhow, water under the bridge!

  6. […] is a great explanation by Harbs of why it’s so hard to save back to earlier versions in InDesign. He’s […]

  7. Mike Rankin says:

    Thanks for writing this! I was going to write something similar on InDesignSecrets to address the never-ending “downsave” issue. Now I don’t have to write it. :)

  8. Alan Gilbertson says:

    Thanks for a lucid exposition, Harbs. I’ve had to go over the same issue with people, and this is far more informed and useful than my attempts!

    You are so right about ePub. It’s so radically different from CS5 and earlier that it’s hard to call it “improved.” Rather, it is finally real. I’m not in the least surprised that it was a case of “nuke, and start over.” It definitely feels that way. (@Zev: The first test I did, the export happened so fast I thought nothing had happened. So, yes, it’s fast. And it works in the BOOK PANEL, too. — Be still, my beating heart…)

  9. Zev says:

    Thanks Alan. It looks like they really scratched all the scripting code and started over in C++.

    I think they originally did it with scripting because they had their doubts about the success of ePub. Once iBooks was launched ePub took off which required Adobe to go back to the drawing board.

  10. Martin Diers says:

    I’m sorry, but I don’t care what a “little birdy” said. This is total BS. There is one reason, and one reason only that Adobe revved the file format rather than retaining the CS5 format: Forced upgrades.

    As a print shop, I am now starting to receive files from a couple customers in 5.5. “I’m sorry, but could you export that as IDML” is not something I should have to be saying to my customers this soon after I upgraded to CS5. Adobe knows this very well. By revving on what really does amount to a point upgrade, they know they are forcing customers to upgrade early. It equates directly to more revenues.

    Had they made a cheap upgrade for CS5 suite owners, I’d give them a pass. But they didn’t make it cheap. I have a lot of CS5 seats, and I wasn’t expecting to upgrade until CS6, and a reasonable amount of time has passed.

    This is a total dirtbag move on Adobe’s part.

    • Harbs says:

      Your conclusion presupposes that the decision to change the file format was made by marketing and not engineering.

      You are of course entitled to your opinion, but I happen to know a number of the members of the InDesign team personally, and I really don’t believe this to be the case. This was a strictly engineering decision and not a marketing one. (I’m not even going to debate whether “forcing users to upgrade” is a valid marketing decision or not.)

      On an aside: I owned a print shop myself until recently, and I did not encourage my clients to bring in native files. Why don’t you suggest to your customers to bring in PDFs? IMHO, playing with native files is just a source of headache for both parties…

      • Hi Harbs

        Great article – thank you for the detail on the database side of it.

        I am sure you’re right that this was an engineering led improvement – but I think Martin has a point – Engineering and Markeitng needed a bit more joined up thinking. Ultimately improvement is done to please the customers… in which case the customers would be much more pleased if this was offered to them at a fairer more balanced price. It might well have been a huge job to rewrite this, but is that our fault? I run a product development company so I know it takes time and a few iterations to get things right. But it’s not right to force the customer to pay top dollar for what is basically a fix. They don’t seem to understand that Creative Suite it is quite a big investment for companies in the first place, making a major commitment to Adobe. Therefore further incremental improvements where you are forced to pay almost as much again as new software is really quite insulting. This is a .5 upgrade because it is exactly that… a .5 upgrade. It only delivers some limited improvement. So WHY does it cost more this time previous full upgrades? In a fairer world (where Adobe would recognise the heavy responsibility weighing on them as the only supplier in the market), Adobe would consider what the actual extra value of this upgrade is to a company and just charge that. For us it might shave off an hour or two a year (per seat). Maybe. But there’s the hassle of purchase and installing a new version to balance against that. So the true value to us of Design Premium CS5 -> CS5.5 is maybe £50 – £100 tops.

        If they charged £50-£100 then they’d have a lot more people upgrading no doubt. Instead of begrudging annoyed customers with a rapidly diminishing sense of loyalty, they would have happy loyal customers. The world would be a better, brighter place – with a feeling that massive mighty corporations are actually care and are on our side. Which might even mean that people get up with a bit of extra pep… go do more business… buy more seats of Adobe CS. Which is presumably what the engineers would want? Hey but what do I know. Perhaps I am just a freaking lunatic.

        • Harbs says:

          Hi Andrew,

          You make good points — ones that I agree with.

          I also feel that the upgrade price to CS5.5 is high.

          Quite frankly, the issue of down-saving is most felt by service providers. Adobe used to have a program for service providers where it was possible to get CS software for a greatly reduced price (free? I don’t remember the exact details). Unfortunately someone at Adobe decided in recent years to cancel that program. In my mind that was a travesty, but then again, what do I know about how Adobe makes their decisions… 😉

          If the Service Provider program was still in place, the pain of this issue would be greatly reduced.

          In lieu of the service provider program, what I would personally do to support outside files would be to purchase a single stand-alone InDesign license. When new versions come out, I’d upgrade that separately from whatever suites I had for production. Single InDesign license upgrades are not terribly expensive…

          • Anna says:

            This is how Quark lost the market totally to InDesign. A few years ago everybody was on Quark. We, the workers were devastated when we had to give up Quark. But they were just too arrogant and the entire publishing industry switched to InDesign, hard as it was to get everyone in the chain to change. (MUch harder than just getting people to upgrade to a newer version.)

      • John says:

        Harbs, I noticed you said you owned a print shop until recently, I guess you like many, many other in the printing business were forced out of business or got out because there was no money to be made. Does Adobe not get it. The printing business is hurting, printer are going out of business right and left and Adobe does not care. They are forcing businesses to spend more capital, which they do not have or loose customers. Maybe the engineers could come up with some new features that would help printing businesses make more money not spend more money. Get in touch with your customers!

  11. Anne says:

    All I can add is “AAAAAAAGGGGGGGGHHHHH!”

    So annoying, so much mucking around. I too have been using Adobe design products since the dark,distant days of Pagemaker. I love ID, but the work flow between my home studio and the work studio is a mine field — especially working under constant press deadlines. Please Adobe, read the countless exasperated users on your forums and fix this issue. Yes, we love the whole cool interface, “creativity at your fingertips” marketing hype, but all, most designers want, is get the job out, looking amazing, with as little amount of hassle as possible. My .5 upgrade is a fizzer now as I forgot my employer has only CS 5 and won’t be upgrading anytime soon. Please pass me a hammer…. sigh.

    • Harbs says:

      I feel your pain…

      Two comments:

      1) Pagemaker had the same issue with back-saving.
      2) I’m not sure that this will help you, but if you purchase TLP licenses, it’s possible to request a downgrade so you could work in the same version as you employer.
      This can not be done from a boxed version — only from a TLP license (site license).

      If you just purchased your copy of CS5.5, you can try to talk to a representative about returning it and getting a TLP license plus downgrade instead.

  12. Anne says:

    Thanks Harbs,

    Can you please explain fully what a TLP license is? I see you refer to it as a site license. I presuming this means purchasing the software online?

    Regards from New Zealand! :)

  13. Harbs says:

    I believe TLP stands for Transactional Licensing Program.

    You need to purchase TLP licenses from a knowledgeable representative (either at Adobe or reseller). TLP licenses are issued in the form of a certificate with the details of the licensee and the product(s) being licensed.

    HTH,
    Harbs

  14. Jenny says:

    Do both InDesign 5 and 5.5 files use extension .INDD?
    When exporting 5.5 files to be opened in 5, is the extension .IDML?

  15. Paul says:

    I’m all for progress but I think Adobe would have served us better if it created an entirely new application that didn’t effect the folks doing mostly print. As it stands, I don’t know one creative who uses the entire functionality of CS3, 4 or 5. Not counting the earlier versions. I dumped Quark along time ago because of their arrogance. I don’t think it’s fair and I do believe we are getting pinned to the wall.

  16. Zaid Al-Hilali says:

    Good information Harbs… thanks.

  17. Robin says:

    This is all fine and dandy but would if you don’t have the CS4 version to save the IDML version? We went from CS3 to CS5.5 and we cannot save a document backwards. Any suggestions?

  18. Rosi N says:

    This is great overview and was very helpfull when i first time faced the problem but it was simple – how to save a file by CS5 so a colleague with CS4 to keep working with them.
    But now I have a problem – a client has catalogues in CS5 and their translations agency work in CS4. Catalogues are book organised and everyone contains 30-40 files so opening each one saving as idml, opening in CS4, saving in CS4, adding to book and looking for problems takes really long time.
    Is there any short cut to make this faster!
    Thank you in advance.

  19. Courtney says:

    I have come across an issue where I have several files that were accidentally saved in CS5.5 but the freelancer that I have to send the files to only has CS4. I exported to idml but found when the files where opened in CS4 some of the items that were on the master pages that had been overridden on the actual pages to have the correct text applied to them where unexpectedly reapplied to the pages. So now I have the ones with the text (the correct ones) and one behind it with the default FPO text. In order to meet a date I had to manually go through over a 100 pages and removed the duplicating elements from there masters and then recheck all of the pages. Does anyone have any idea why this is happening? (there was no time for upgrading during this issue)

    • Harbs says:

      It’s hard to say, but if the master pages were reapplied for some reason, that would cause duplicates on any items that were overridden (such as text frames with local content).

  20. Lothar says:

    I very much appreciate the technical explanation, but I think the “conspiracy” stands because when Adobe releases a new version, even a .5, they quickly make it really difficult to get the previous version.

    So if I have a small business with 10 designers on CS5 Design Premium and I want to add an 11th designer today, I have to buy CS5.5. Now my other designers are having compatibility issues with the new guy. So I consider sucking it up and upgrading all of them to InDesign CS5.5 at $120 a pop. Nope. Adobe doesn’t let you upgrade individual titles within a Creative Suite package. So instead my only option is to get 10 copies of the CS5->CS5.5 Design Premium upgrade at $400 a pop when all I needed was InDesign CS5.5.

    Now I’m looking at spending $4000 on top of the new copy of CS5.5 just to maintain compatibility. If Adobe would let me purchase those InDesign CS5.5 upgrades or just purchase CS5.0 to maintain compatibility, I’d drop the accusations of conspiracy, but this just feels like they are squeezing their good paying customers for all they’re worth.

  21. R says:

    This is still very frustrating. We recently upgraded the whole office to CS5, added a new employee a few months later and could no longer buy CS5, but had to get CS5.5. Now we can’t access any files our new guy touches. The whole IMDL thing is just plain annoying. If they are going to not allow backsaving in the same point version at least give us the option to buy older versions so our whole shop can be on the same version.

    *end rant of a dissatisfied Adobe customer*

    • Harbs says:

      If you get TLP licenses (not boxed versions), it is possible to get a downgrade so everyone can work on the same version. A competent reseller should be able to help you with this.

  22. Frank says:

    Everyone needs to write Adobe about this and express their opinion. To me this is unacceptable.

    http://www.adobe.com/support/contact/

Leave feedback