Zipping Files Fun

Clemens Helm speaking at viennaJS in May, 2017
222Views
 
Great talks, fired to your inbox 👌
No junk, no spam, just great talks. Unsubscribe any time.

About this talk

Learn how to zip files on the fly in the browser and on your server. You'll see how easy it is to put remote files into a zip archive and streaming parts of the archive to the client at the same time.


Transcript


- Hi everyone. Today we're gonna have fun zipping files. My name is Clemens. That's me in real life. I'm the Dev Lead of ChillBill. At ChillBill, we automate accounting for small companies and we play around with a lot of fun technologies like Meteor, React, Machine Learning algorithms, and JSZip, which I'm gonna introduce today. JSZip is a library for creating and reading zip files. So basically you can insert files and make a zip file out of it, or you open a zip file and read the files out of it. It works on a server and on a client, means in the browser. And everything is really asynchronously and it's really nice to use. I'm gonna show you some code. I built a little application. I'm gonna show you the link afterwards. It's on GitHub. The application is called Zipedizip. That's what it looks like. Maybe I can zoom a little bit. Awesome. Okay, so we've got two scenarios here what we can do. On the one hand, we can choose some files and make a zip file out of them. And we can also choose a zip file and extract the files out of it. I'm gonna show you the code first just in case that it doesn't work. This is the part where I read files, where we kind of select files with a file input field, and put them into a zip file. So what's happening here is ... By the way, I use a lot of this new fancy JavaScript syntax things, so if some of this is new to you, or you think, huh, what's happening here? Just shout, and I'll do my best to explain it. So what's happening here is on this input field where we select our files, we listen to the change event, so if the files are selected, we get kind of an array of files. It's not really an array, it's a file list. And then we take this file list with this, my syntax thing here, we convert it into an array so we can iterate with Map, or the list of files. What we want to create is an object with the name of the file, the file name that entered in our computer, and the content of the file. But we get this content as a promise because what we use for reading the file is a file reader, who knows a file reader? Awesome, I know that since today when I prepared the talk, file reader is not supported by a lot of browsers yet, that's why I, no, file reader is supported, file saver isn't supported. File reader gives us the possibility to read files that we select with a file field in the browser and gets the binary content. And what we're doing here in this file reader, in this read file method, is we make a promise, we make a file reader, and we say, when the loading of the file has finished, so on load-end, and the state is done, probably, there are other states like failed, or whatever. Then we call the resolve function of the promise with the result. And the result will be a binary string of the contents of the file. And to initialise reading this file, we have to call reader, read as binary string, and we pass the file that we pass here in this read file thing. I think I'm going to show you how it works, it makes it maybe a little bit nicer. Okay, I've got some pictures here of my three year old son, with his pants on the head. I'm just going to select two of them. Say, Open. Okay, and what's happening here, you can't see it, so you have to trust me. What happened was ... Zipedizip, we downloaded a zip file, and the zip file contains the two images that I selected. So how did this work? We already saw, okay, we're reading in these files with the file reader, and here we create for each of these files an array of object with the file name. The file name still is the same in this downloaded zip, as we can see here, and with the content promise. Okay, and this is where JSZip actually comes into play. We just make a new JSZip object, and for each of these files that we just read, we say, zip file with this name and this content promise. And the nice thing is that we can actually pass promises to JSZip, we don't have to say, this is the name, this is the content, everything needs to be resolved in advance. But we can just give it a promise and it will wait until this promise is resolved to build this zip file iteratively. And at the end, we want to save the zip file on our computer, so we say, zip generate async, which just means, build a zip file of type blob. And when this is done, then save this blob in the file zipedizip.zip. You might be wondering where does Save As come from? That's another thing you're gonna learn today. There is a file saver object, or library, that's actually part of the JavaScript standard, but no browser supports it yet. So we have to use an external library which gives us the Save As function, but hopefully soon we won't need it anymore. But, it's awesome because you saw it just acts like a download, but in fact, we build the whole file in the browser. Just say, Save As, and it will save it on our computer. What else can we do with JSZip? We just saw, okay, we can select files, but that's not really that useful because the files were on our computer before. They're in a zip file on our computer afterwards. What else is there cool, what else cool is there to do? We can just, for example, write a file as a string. We can write a string into a file. So, as I call this Magic Spell, probably could have just copied it from below. So, I'm gonna save this Magic Spell into this Magic Spell TXT file. Then, I reload in the browser, just the newest version, to trigger the zip file I still have to select a file. And when we open this zipedizip to unzip ... It contains also the Magic Spell TXT with our magic spell inside. You don't seem very impressed. Okay, but there's even more. We can even create folders inside of our zip files. So we could say here, Folder, text, for example. And then just chain this file call afterwards. So when I do this whole thing again, and unzip it ... I have no idea why it goes to this screen, always ... Then there's the text folder with the magic spell inside. - [Student] Love it. Yes, give me some feedback. - [Student] Yeah, amazing. - Awesome. But actually, what would be even nicer, was if we could fetch some remote files and build them together into a zip that we store on the client, right? This would be kind of a real world use case, that probably every one of use could use. So I'm not gonna pretend anymore that I'm inventing this, I'm just gonna put this here. Okay. Whoops. Okay, so what's happening here? JSZip gives me this JSZip-utils thing, which allows me to get binary content easily, and it has a callback, and in the callback, I say, Zip that file, we want to save as logo JPEG, the data that we downloaded from the logo URL, which is images/logo.jpg. What happens when we try this? Oh, no! It didn't work! What happened? Has anyone any idea why it didn't work? - [Man] Data - [Student] Well you just generated the binary content, so you have it twice, am I right? - Yeah, the problem is that actually ... - [Man] You have to generate it inside the first promise, right? After you get binary content, then you generate - I got the binary content, the problem is that actually, here, it isn't resolved yet, so I think ... No, actually, this is called, yeah, that's it. I just remembered. The zip.file is called after, this is called. After the Save As is called, because it's called asynchronously, right? It downloads, and when a download is finished the zip file is already saved. So it's going nowhere. So what can we do about this? We use more code. No, yes. Okay, so, I'm wrapping this in a promise, and I say, you remember, right? JSZip supports promises and it waits for the promises to resolve. So, I make a download promise. I use the same code that I used before, here, get binary content, logo URL. And as soon as the download is finished, I call the resolve function. That means, inside of JSZip, this promise resolves, and everything is fine because the zip file is only saved after this one promise resolves. So, when we try this again, was that clear? - [Man] Yup. - Yes? No? - [Student] Doesn't get binary content return a promise, if you don't specify the code? - Pardon? - [Student] Like, you say JSZip supports promises? Why would you wrap get binary content in a native promise? I'm wondering if this binary content doesn't just return a promise. - I don't know, we could try it. That's a good point, actually. Let's do it my way first and then we'll try yours. Okay, we've got another zip file. And there it is! The logo JPEG, yeah! Okay, but I'm gonna give your approach a try, as far as I understood it, you mean we don't need this promise here. We can just move this up here. Get rid of this. - [Woman] Comes your error data as well. - Ah yeah, exactly. And then move this into, no. - [Man] No, just remove the callback, entirely. I'm just guessing here, but ... - Ah yeah, right. No, one more. Yes. Okay. Reload. - [Student] Whoa ... - Whoa, yeah! It really is getting exciting, yeah, it's here! But it's ... No! Okay, so it works half. My version works better. Okay. Is there anything else that I've got here in my cheat sheet? No, I don't think so. So, now the other question is, how can we do it the other way around, right? How can we take a zip file and extract the contents? And fortunately, I also prepared something, which is this file and it's even shorter. You will love it. So here we're doing kind of the same thing. We're listening to this other input field, which you might have seen, which allows us to select the zip file. And we also use our read file function again that I showed you before, which uses this file reader stuff. Then we just say, JSZip, load async the zip file. And once this is done, the zip file is ready. Then we can iterate over each file in the zip file. I just push all of the paths of these files into an array, which I called contain file paths, and then I make them look nice. So I create list elements of them and then I put the list elements into an unsorted list and put them into another HTML element. So ... When we take a zip file now, will it work? It works! So yeah, we can try a few more. Of course, this is not really the, the use case that you might get excited about. But you know what you could do with it, you could also get the binary content. You could use it, for example, you let a user upload a zip file with images. You display the images in the page right away, without having to go a round-trip to the server, unpack it, then load the images in the front-end and stuff like that. You can just do everything like that in the browser. That's super awesome, I think. That's about it, from the live demo. I've got a few links for you. This is the URL of JSZip, it's an awesome package. At ChillBill, we use it in a totally different way than I showed you today, but I'll get to that. The code of this example, you'll find on my GitHub account, clemenshelm/zipedizip. And if you want to be friends with me you can go to my Facebook profile. And if you want to check out ChillBill, of course, that's also our company URL. Just one more thing, I lied. When you read the immediate description of the talk it said, I will show you how to use zipping files on the client-end to server, and this was not true. I decided spontaneously only to show you the client today, and to go into more detail in the live coding session. At ChillBill, we use it actually only on the server. Which is also super awesome, because you can do things like, you download a lot of files from external sources, you get promises back, you put all of these promises into JSZip, and JSZip will take care of resolving them. And as soon as one promise is resolved, it starts streaming the zip file to the client. And actually this is a really, really nice approach of zipping files on a server, which is very memory-efficient, and has super user feedback because the download starts right away, and people don't have to wait until everything's downloaded on the server, and then is zipped, and then the download starts, stuff like that. If you're interested in that, come talk to me afterwards or go to Roland and say, "Roland, please, "make Clemens give one more talk on JSZip on the server." And then we'll see each other again in a month, or two, or three. Thank you very much.