June 22, 2019
I was at the gym yesterday - the gym closes at 10 PM. I got in around 9:20 PM, and did a 3 mile run. I don’t know what it is for me, but I feel like I go through a cleansing every time I go on a run. It’s great.
But that’s not the story. I was at the gym, I go through the shower and everything. I’m in a rush to get out because I feel sorry that I’m making the workers stay late because of me. I run out.
The minute I leave the gym doors I check my bag for all my clothes, and I realize that I left my underwear and shorts in the shower. The doors lock after closing time. I screwed up.
So I’m hoping the store managers would come back to the front desk near the door.
I time-box myself to wait 10 minutes. They don’t show up in the front. I start my walk back home, cursing underneath my breath to have slowed things down and double checked rather than rushing out, because… the workers also need to go home! It’s been a long day for them. The clothes I left may or may not be there tomorrow - the cleaner may just throw out my dirty clothes. I doubt there is a lost and found where they keep all dirty gym clothes that are festering with germs and sweat.
At this point, I realize two things.
- There is a backdoor to the gym. That’s why the employees never came back out. They must have a back entrance they prefer. This can mean a lot of things - I really think that there is a backdoor for almost everything. When going to a concert, trying to meet someone you really admire, or even a job application. There is definitely a backdoor somewhere to meet them. You just have to find it and crack it open.
Once the door shut behind me, gym time was done. No turning back. It’s over. Done. Game Over. Gym Over. There’s no point for me to wait there until the cows come home. What’s done is done, and it’s time to move forward. What I’ve been struggling with is the fact that during the past few months, I never took initiative and got to know someone I was interested in until they left. There’s been a lot of regrets on that.
But what’s done is done. There’s a reason why we don’t have eyes on the back of our heads. We move forward. We don’t let what’s behind us bother us for every step we take. Every second we spend looking at the past, we sacrifice seconds that could be spent moving forward.
Time to move forward!
P.S. I went back to the gym the following day. Fortunately, the clothes were on top of the shower curtains, and the cleaning crew probably failed to notice. I got my clothes back!!! =D.
First drafted May 11, 2019.
June 13, 2019
During my time in downtown Brooklyn, one of the things that drove me mad was looking at restaurant reviews. Friends and I would decide to go to restaurants, and we’d look at the ratings. When we did, we’d get an aggregate rating of 3.5 or 4. But what does it all mean? It really doesn’t mean anything unless we understand the distribution of the data.
It drove me nuts. I had to find out the average, and what the review landscape look like.
So I did. I cobbled together a program in Python using Scrapy, Pandas, and Matplotlib. I would have left out Sir Scrapy, but the review website’s api had this random feature where if you queried a restaurant for all its reviews, it would give you 3. Grr…
I used the a certain restaurant review website’s own filters to hone in on downtown Brooklyn within a mile radius. The website’s api request gave me two two longitude latitude pairs. They are shown below:
Interesting. So it’s either endpoints of a circle, or a rectangle. I think it’s a rectangle, so we’re going to refer to this area as the rectangle.
This search query gave me a starting point of all restaurants, but displayed only 10 or 20. I scraped them, and then used a program to move to the next page. I stored the set of restaurants in a directory. Then for every restaurant, I automated a http request, got to the main webpage which was an anchor point of a restaurants set of reviews. So I went through all those reviews and got them too.
Number of restaurants in the bounding box: ~500.
Number of restaurant reviews: ~70,000.
The average of all review ratings: 3.70
Standard deviation of all ratings: 1.32
So people in Brooklyn tend to rate 4s and 5s much more often than 1s and 2s. I wonder if you do this for every city, then could you gauge a friendliness metric for every city, and see if it correlates with the happiness ranking of every country? That would be AMAZING.
This is if you plot average rating of restaurants, and plot them by rounding them to the nearest 0.5.
This is simply the number of reviews and their restaurant names. Seems like it follows some power law distribution, but I’m not quite sure. It may be just one outlier.
Top 20 restaurants by review count:
I guess this can be interpreted as popularity or to some degree how much people care. One can perhaps use this information to extrapolate the length of time the restaurants have been around.
||Number of Reviews
|Rocco’s Tacos & Tequila Bar
|The River Café
|Mile End Delicatessen Brooklyn
|Vinegar Hill House
|Alamo Drafthouse Cinema Downtown Brooklyn
|Two 8 Two Bar & Burger
|Dekalb Market Hall
Top 50 Highest Restaurants by review average (ignore just 5s – probably insufficient number of reviews. Perhaps I will add review count next to the rating later…)
|Bird’s Eye Vietnamese
|New Fresco Tortilla Plus
|Smith Gourmet Deli
|Thai on Wheels
|Lillo Cucina Italiana
|GMC Temaxcal Deli & Grocery
|Simple NYC-Downtown Brooklyn
|Yumpling Food Truck
|Pret A Manger
|Dariush Persian Cuisine
|Grand Canyon Restaurant
|dot & line
|Rice & Miso
|Yemen Cafe & Restaurant
|ACE Thai Kitchen
|Bread & Spread
|Forcella Fried Pizza
|Doner Kebab NYC
|W XYZ Bar
|Shawarma & Grill
|Daigo Handroll Bar
|Sultan Restaurant & Cafe Lounge
|Downtown Natural Market
- I need to ask someone who is really knowledgable in statistics, or do research on if there’s a law that correlates the number of reviews to its true rating. What should it converge on? For example, a review with 1000 reviews with a 3.5 should be weighted differently with a review with 10 reviews with a 3.5.
- It would be interesting to plot the average words per review or aggregate words per restaurant.
- What would be the most commonly used and the most least commonly used words? I want to run it through a basic NLP program to stem the word, remove stop words, etc.
- What if we have a list of words, give them a raw numerical value of positive and negative numbers, and average that out. Would the rankings be different?
- What if you plot out the datestamps of the reviews, and use some metric for happiness/economic activity, and see if there’s a correlation between that and the stock market or the consumer sentiment index? Would there be a correlation?
- See what another review service from a search engine’s data looks like.
Examining all this data is a lot of fun! But for now, my experiments are on hiatus. There’s already enough things to read and build! But if anyone wants some Brooklyn data, let me know!
- You should probably use this dataset to get interesting insights: https://www.yelp.com/dataset/download
- DO NOT SIMPLY SCRAPE ON YOUR LOCAL NETWORK. If you mess up, it’ll cause your ip to be banned. Your ISP will refresh your ip within a month, but it’ll annoy the people who share the internet connection with you. Either use a vpn, or spin up a cloud computing instance and use that to download your data.
- Rate limit your scraper. Don’t try to download all this data within a span of a minute. Pace your scraper to download a reasonable amount in a minute so you don’t overload their apis. Or get caught. There is no need to rush.
- You should learn regular expressions. I was talking to someone who was doing data collecting with DOM traversal. That maybe more painful than building a reasonable regular expression. My two cents.
June 11, 2019
Humbled. Today is the 11th day of June of the year 2019. I’ve felt this somber melancholy that I really don’t know anything. Compared to the vast knowledge that have been built, and will be built, I know so little. There’s no need to be giddy that you’ve solved problem X, when it’s one among the infinite ones out there. Similarly, maybe I shouldn’t be extremely stressed out about mistakes and mess ups, when those actions are cosmic dust in a spectrum of all available decision. I reflect back on the younger days, when I thought I knew more, and I was wiser than someone else. There’s some shame in that. I feel like an idiot for ever thinking that way.
You think you understand, then you take a second look. And you’ve realized that you’ve never really internalized nor grasped. Whether that be a simple story or a math equation, I feel that there is always deeper meaning underneath it all. It’s like scratching the surface of an iceberg. You can keep going down forever.
It’s a two sided coin. On one hand, you will never run out of things to learn, explore, and think about. How exciting! But on the other hand, you will never learn everything. Impossible.
Other things I struggle with:
- You can call someone out for their bs, but I find myself in similar situations and under the same conditions I can easily make the same mistakes. I.e. criticizing someone for not willing to do something then a week later I find myself not willing to do something easy for a stranger because I was in such a terrible mood.
- It’s hard to strike up conversation with people when you know that the relationship will never be deep. What’s the point of striking up a conversation with a person at the subway? You may never see them again. But then again, you may never know where conversations end up. They may offer you a book recommendation, or something super insightful and change the way you think. But you don’t want to annoy them either.
- Making good choices. For every decision in the decision space, there is a theoretical best decision. For yourself, there is a theoretical best decision that you can take at every time t. I want to take those optimal decision, or decisions close to them. This is hard. The big recommendation I keep hearing is to study philosophy deeply. It has characteristics that cannot be defined my structured math and logic.
- Becoming a more efficient coder. One of the things I’ve realized I need to work on, still, is coding a little and testing a little. Knock on your steps before going to deep. Dipping your toe before jumping into the ocean. Not digging myself into a cobble-web of complexity.
- Becoming a better communicator. Lately, I feel like I’ve had a string of.run on sentences. People can’t keep track. I should cut, verify they understand, cut, and perhaps get more feedback. Usually shorter sentences are better than longer ones Ugh. But at the same time, it annoys people if you keep asking for feedback. Maybe the better approach is to study people’s so I can detect confusion.
Drafted June 11th, revised June 13th.
June 08, 2019
Interesting stuff always happens on the New York Subway.
I saw this kid - maybe a middle schooler selling fruit snacks on the train. He went around the cart saying,
“Excuse me everyone. I have something to say. I’m an honest man trying to make a living. I’m not here to get your money to do drugs or anything like that it. I’m fundraising for <something I can’t remember>. Two fruit snacks for a dollar.”
It made me smile. He went around asking the cart if anyone would like fruit snacks. He wasn’t just standing there - he was actively looking for new customers.What a boss.
I wanted to buy one, but I didn’t have cash on me =(. Nothing. Nada.
When I see things like this, I wonder how much this potential this kid has - he’s doing something that other kids, even adults like me have a hard time doing. He’s also putting his own spin on it instead of just saying fruit snacks for a dollar. He’ll definitely be really successful if he gets the right mentorship and nurturing. There’s so much potential in kids like this. And fortune favors the bold. I could learn from him, and man, if I was a teacher I’d want an awesome student like him.
Originally drafted May 12, 2019.
June 07, 2019
I read this gem in the past month from Pascal’s Penses. This is his snippet on love:
“What is the self? A man goes to the window to see the people passing by; if I pass by, can I say he went there to see me? No, for he is not thinking of me in particular. But what about a person who loves someone for the sake of her beauty; does he love her? No, for smallpox, which will destroy beauty without destroying the person, will put an end to his love for her.
And if someone loves me for my judgement or my memory, do they love me? me, myself? No, for I could lose these qualities without losing my self. Where then is this self, if it is neither in the body nor the soul? And how can one love the body or the soul except for the sake of such qualities, which are not what makes up the self, since they are perishable? Would we love the substance of a person’s soul, in the abstract, whatever qualities might be in it? That is not possible, and it would be wrong. Therefore we never love anyone, but only qualities.”
Wow. One can interpret it as the fact that we never love anyone - only their qualities. It’s a very logical, cold, reductionist point of view. It kind of reminded me of principle component analysis - how you extract information to the most relevant features that explain the most.
Most qualities of a person are malleable. People change. A person can be selfless today, but tomorrow they can be selfish. They can be the nicest person you know today, but something tragic happens to them tomorrow, and they become the most bitter and cynical person you’ve ever known.
This quote kept me up at nights before bed. Is it true that we really never love anyone, except for their qualities?
After much thought, I lean towards no.
One can argue that there are qualities of things that do not change no matter what. No matter how I change, I will still be my mother’s son. No matter how I change, I will still be me. Maybe there are essential to being me, that stand the rigor of time.
Secondly, let’s take a step back and evaluate this scenario. If someone asks me, “Why do you love your brother so much?”
I can list out all the qualities of why I love my brother. he’s smart, funny, kind, and thoughtful. Hmm. But what if he’s dumb, full of lame jokes, mean, and thoughtless - would I still love him? I think so. Aren’t there siblings we love that are like that already? There’s more behind than simple qualities.
Similarly, if I love someone, I can list out the qualities of why I love that person, but I love them because of who they are. Maybe listing out qualities is one abstraction level down to explain things that you cannot explain. If someone asks you, “Why do you love Jane?” Wouldn’t the optimal answer be, “Because she’s Jane. There’s no one like her, and there will never be anyone like her. Ever.”
Also, love and building affection is a gradual process. People grow on you! So suddenly one of their qualities change, and if you decide to cut them off, maybe there wasn’t any substance in the first place. After all, you can’t love someone you don’t truly know and without understanding who they are.
The other side of change
I took a very negative approach to thinking about this in the beginning and focused on how people can change for the worse. But doesn’t Pascal’s thought apply in the reverse direction as well? People can change dramatically for the better. You can be bitter enemies with someone and abhor spending time with them because of certain traits. But everything is subject to change. They can change in ways you can’t imagine. Who knows. Maybe you two become besties later on? Similarly, you’re also subject to the forces of change. Maybe they didn’t change to become more likable - you did!
Perhaps people are like quarters. I mean the coins. When we examine one side, we see an engraving of George Washington. Flip it over, and we see one of fifty states. Can you really predict this? You can’t predict which state, nor can you predict based on the picture of Washington that you’re going to get a state on the back. For better or for worse, no one can really know for sure how people change.
Update 8/1/19(in progress):
I talked to someone my friends on this quote and asked what they thought about it. To my surprise, when I asked my friend (whom we shall refer to as Roy) he was the first one who said he’s thought deeply about this before! Woah.
Here is what I got from my friend that was an interesting idea.
He said that he doesn’t believe in true love. Here’s his argument.
For every personality, there is only a finite number of personalities. Given how many of us there are, there are bound to be cases where two people have the same personality. The Pigeonhole Principle!
Here’s a diagram(also to be added later):
After all, Confucius said, “All people are the same; only their habits differ.”
To some degree, that is true, but it’s only one way to look at the picture.
I asked him if he believed in a soul - he said yes, but it has nothing to do with personality. Hmm…
Last updated August 2nd, 2019