Growing Pains from Development to Production

This week in code has been mostly a story of struggle. I’ve hit a complete wall twice, and I’ve somehow made it out the other side. I never imagined that all of the code that I worked so hard to perfect in the development environment would fail me in the live environment. The issues have been minor, but still hard for me.

Change is hard because it is so dramatic. Seedlings emergin photo by Christian Joudrey on Unsplash

I want to share two major hurdles that I hit this week and how I got over them. The first is with Stripe redirects and the second is with user uploaded images.

Point your stripe redirects correctly

Stripe is a great way to handle payments. I love how they have a sandboxed testing environment where you can use sample accounts and make sure your whole process works before you even setup your bank details. There are some things you may not catch if you are moving from development to production, at least for the first time.

Essentially, anything that you setup through the dashboard for your development environment needs to be recreated for your production environment. This seems simple unless you forget that you set up a redirect URI in the test environment. Or, that you created subscriptions in test but not live. I spent two days on back and forth discussions with the helpful support folks at Stripe before I realized that I had been able to parse an exposed URL in the testing environment that I could not parse in the live environment. Changing my redirect URI through the Stripe dashboard to my API endpoint that would process the tokens was the final solution, but it seemed like it was a much bigger deal based on the errors.

Alternatively, you can setup your Stripe account features through the API.

You cannot use multer to upload to an uploads folder on heroku

Do you have code that looks like this:

routes/things.js

const express = require('express');
const router = express.Router();

const multer = require('multer');
const storage = multer.diskStorage({
    destination: function(req, file, cb) {
        cb(null, './uploads/');
    },
    filename: function(req, file, cb) {
        cb(null, new Date().toISOString() + file.originalname);
    }
});
const fileFilter = (req, file, cb) => {
    if (file.mimetype === 'image/jpeg' || file.mimetype === 'image/png') {
        // pass
        cb(null, true);
    } else {
        // reject
        cb(null, false);
    }
    // send an error cb(error, true); etc.
};

var MAGIC_NUMBERS = {
	jpg: 'ffd8ffe0',
	jpg1: 'ffd8ffe1',
	png: '89504e47',
	gif: '47494638'
}

const upload = multer({
    storage: storage,
    limits: { filesize: 1024 * 1024 * 5 },
    fileFilter: fileFilter
});

const middleware = require('../middleware');
const controller = require('../controllers/thing');

router.post('/:thingID/finished', middleware.isUserLoggedIn, upload.single('upimage'), controller.finish_thing);

related form, let’s call it views/things.ejs

    <form action="/things/<%= thing._id %>/finished"  enctype="multipart/form-data" method="POST">
        <input type="file" name="upimage" />

            <div class="form-group">
                <label for="inputdesc">Describe this thing:</label>
                <input class="form-control" type="text" name="description" id="inputdesc" placeholder="This is a thing">
            </div>
            <div class="row justify-content-center mx-md-3 px-md-5">
                <div class="col text-center">
                    <button class="btn btn-success">Save Changes</button>
                </div>
            </div>
        </form>

I tested code like this several times in my development environment. I thought it was fine. However, it resulted in a 404 error for my users. I am deployed on heroku, and I think that a bigger problem could have occured if users were able to upload and then my site refreshed, their data would be lost.

The solution that I followed was to create an s3 bucket with amazon, and setup the image upload form to interact with amazon.

The router loses the multer code and gains some code to communicate with the s3 bucket.

Note: One item in the controller needs to change as well, instead of found.image = req.file.path;, we now use found.image = req.body.upimage; when saving the thing to our database. This is where we are saving the url of the image.

routes/things.js

const express = require('express');
const router = express.Router();

// add in stuff to connect to aws
const aws = require('aws-sdk');
const keys = require('./config/keys');
aws.config.region = 'us-east-1';
const S3_BUCKET = keys.S3_BUCKET;

const middleware = require('../middleware');
const controller = require('../controllers/thing');

router.post('/:geID/finished', middleware.isUserLoggedIn, controller.finish_thing);
// removed the multer middleware

// add another route to get a signed s3
router.get('/sign-s3', (req, res) => {
  const s3 = new aws.S3();
  const fileName = req.query['thing-name'];
  const fileType = req.query['file-type'];
  const s3Params = {
    Bucket: S3_BUCKET,
    Key: fileName,
    Expires: 60,
    ContentType: fileType,
    ACL: 'public-read'
  };

  s3.getSignedUrl('putObject', s3Params, (err, data) => {
    if(err){
      console.log(err);
      return res.end();
    }
    const returnData = {
      signedRequest: data,
      url: `https://${S3_BUCKET}.s3.amazonaws.com/${fileName}`
    };
    res.write(JSON.stringify(returnData));
    res.end();
  });
});

Major changes also happen in the form, which now has an image preview (bonus) and javascript to show a preview of the image and to upload it to the AWS bucket. I mostly followed the helpful guidance from heroku on setting up s3 uploads.

	<div class="col-sm-12 col-md-3 image-cover">
            <img id="preview" src="/images/noimage.png">
        </div>
    <form action="/things/<%= thing._id %>/finished" method="POST">
        <input type="file" name="upimage" />

            <div class="form-group">
                <label for="inputdesc">Describe this thing:</label>
                <input class="form-control" type="text" name="description" id="inputdesc" placeholder="This is a thing">
            </div>
            <div class="row justify-content-center mx-md-3 px-md-5">
                <div class="col text-center">
                    <button class="btn btn-success">Save Changes</button>
                </div>
            </div>
        </form>

<script type="text/javascript">
    const thing-name= $("#thing-name").html();

    (() => {
        document.getElementById("file-input").onchange = () => {
            
            const files = document.getElementById('file-input').files;
            const file = files[0];
            if(file == null){
                return alert('No file selected.');
            }
            getSignedRequest(file);
        };
    })();
    function getSignedRequest(file){
        console.log("Getting signed request", supge)
  const xhr = new XMLHttpRequest();
  xhr.open('GET', `/things/sign-s3?file-name=${encodeURIComponent(file.name)}&file-type=${file.type}&thing-name=${thing-name}`);
  xhr.onreadystatechange = () => {
    if(xhr.readyState === 4){
      if(xhr.status === 200){
        const response = JSON.parse(xhr.responseText);
        uploadFile(file, response.signedRequest, response.url);
      }
      else{
        alert('Could not get signed URL.');
      }
    }
  };
  xhr.send();
}
function uploadFile(file, signedRequest, url){
  const xhr = new XMLHttpRequest();
  xhr.open('PUT', signedRequest);
  xhr.onreadystatechange = () => {
    if(xhr.readyState === 4){
      if(xhr.status === 200){
        document.getElementById('preview').src = url;
        document.getElementById('upimage').value = url;
      }
      else{
        alert('Could not upload file.');
      }
    }
  };
  xhr.send(file);
}
</script>

I hope that these insights help someone else who is feeling the growing pains of going from develpment to production.