Skip to content

This is how we flatten data: flatten.js

Daisho Komiyama edited this page Mar 25, 2021 · 36 revisions

One day somewhere in Canada, imagine you got a visualization task with humongous size of dataset out of nowhere, you probably have to sit tight, wish yourself best luck then try very hard to wrap your head around it. I know it because I did it.

Speaking of my case it was disaster at least at the beginning. I've spent some time just looking at screen, counting how many properties there were within an array of objects with a few hundreds of them. After brief thinking time, I started writing JavaScript based on a shallow plan. This was a bad start.

After a few hours, I ended up having a few functions. Not like the principal I always had in my mind; don't repeat yourself, what each function does was way too specific that makes almost impossible for the functions to be reused in other use-cases. They are so-called one-time-use-only functions. I hated them, even though they get the job done. I also hated myself and circumstances that forced me to produce those trash.

I walked out the office, thinking to go for a jog that night.

My mind was full of programming even during a run. I traced each data value from beginning to end in a function I built in my head. I did it so many times simply due to difficulty; number of data layers was beyond imagination. That said, by the time I finished running, I've got a slight idea of how my functions were supposed to be built. This means I have to rewrite everything. but I was totally excited.

Following day, I woke up a bit early then I wrote this. I name it flatten.js

const data = [
    {
	a1: 'platoon 1 leader',
        a2: 'platoon 1 sergeant',
	a_sections: [
            {
		b1: 'section 1B leader',
                b2: 'section 1B member 1',
                b3: 'section 1B member 2',
		b_sections: [
	            {
			c1: 'section 1C leader',
			c2: 'section 1C member'
		    },
		]
            }
	]
    },
    {
	a1: 'platoon 2 leader',
        a2: 'platoon 2 sergeant',
	a_sections: [
            {
		b1: 'section 2B leader',
                b2: 'section 2B member 1',
                b3: 'section 2B member 2',
		b_sections: [
	            {
			c1: 'section 2C leader',
			c2: 'section 2C member'
		    },
		]
            }
	]
    },
];

function flatten (src) {
	let last = '';
	const array = [];
	const accum = {};
	function recursion (src, accum) {
		let includes = false;
		src.forEach(function (obj) {
			for (let prop in obj) {
				if (Array.isArray(obj[prop])) {
					includes = true;
					last = obj[prop];
				}
				else {
					accum[prop] = obj[prop];
				}
			}
			includes ? recursion(last, accum) : array.push({...accum});
		});
		return array;
	}
	return recursion(src, accum);
}

flatten(data);

//output
// [
//     {
//         a1: 'platoon 1 leader',
//         a2: 'platoon 1 sergeant',
//         b1: 'section 1B leader',
//         b2: 'section 1B member 1',
//         b3: 'section 1B member 2',
//         c1: 'section 1C leader',
//         c2: 'section 1C member'
//     },
//     {
//         a1: 'platoon 2 leader',
//         a2: 'platoon 2 sergeant',
//         b1: 'section 2B leader',
//         b2: 'section 2B member 1',
//         b3: 'section 2B member 2',
//         c1: 'section 2C leader',
//         c2: 'section 2C member'
//     }
// ]

I works well regardless the number of layers the data has. However, it keeps data only from the last array if there are more than one array in an object. I've gotta solve this.

Clone this wiki locally