It’s amazing and/but I don’t know why

One of those notes written under a sudden and noticeable motivation spike. I’ll attribute it to coffee but it’s obviously not the only factor. I had coffee many times in the last week but this feeling came maybe once or twice. Something needs to be attributed and as the author I need to give you an anchor, otherwise you’ll go on a wild hunt.

What am I talking about? Yes. I’m with you on this question. (No, this post is not written by GPT-X.)

Last week was rather stressful. In hindsight, all stress was created by me but for a long time it looked like the world was pressuring me. It’s about the feeling when you’ve committed to do something and then have doubts about the commitment. Doubts are thoughts and they take time, and that time is taken from the commitment. The best case scenario is that you fully exchange thinking into confidence. In the rest of scenarios, efficiency is lower and that “thinking” has a side effect of “time wasting” which when noticed further decreases the efficiency. Since you don’t know who cares you imagine that no one cares but everyone is watching. And they’re judging, because, obviously, no one just watches. And they’re criticizing, because that’s the only way imaginary judges operate. And so you try to come up with a way to make the mob go away. A common approach is to double down on the commitment and press it, push it, drive it to cross the finish line as initially declared. Let the grind begin. Never mind that mental doubt baggage you’ve been collecting on the way; it’s no time to resolve it, it’s time to speed up.

Obviously, it isn’t good to speed up now. Peer pressure creates a steep incline and self-doubt adds rocks to your backpack. It’s a double trouble; not only you’re going to struggle but you’re also going to struggle. Yes, correct, twice. You keep on going because at least you’re going. You should give up but that’s what they want. It’s hard and your legs are aching. And your back! What now? How to? What if?

And then it bursts. You’re back on a straight path through a gorgeous forest and your backpack only contains lead-free tuna sandwiches. You have a full memory of the struggle but the pain is gone. You recall imaginary mob but there’s no way to point where it is, and if it is anywhere it’s probably somewhere behind now. What the fuck has just happened?

This happens to me more than I really want. As a self-retrospective(?) person I try to analyze and learn from these experiences. I have notes on events and various data, and when I think I’ve got it – boom, another one. Obviously obvious conclusion is that as a pattern monkey I like to find patterns and they make me happy. So many obvious things, obviously! But that isn’t obvious and it isn’t a conclusion. It’s also questionable whether that makes me happy. It’s complicated, and that state makes me intrigued and amazed. That’s the happiness of level three.

Agents Bar is in public beta

It’s a huge pleasure to announce, yet it is rather scary. As in title – Agents Bar is now in public beta. Go ahead, sign up, sign in and try it. I’d love to hear what you think.

Why beta?

It just doesn’t feel like a full release. There are only 4 agents with limited documentation. The monitoring isn’t propagated to users, they don’t have access to snapshots… Many features are simply turned off to make sure that we aren’t abused or they don’t make sense without that other feature. However, basic functionality is there, so why not just share it?

What’s now?

We continue. In the roadmap we have adding custom DRL agents, keeping selected version, uploading environments, creating League feature and so much more. But, obviously, there needs to be priority in all of that. We’ll be reaching out to potentially interested parties asking trying to get some feedback. We’re currently thinking about reaching out to universities and researchers to learn how we could help them.

If you are, or you know anyone who is interested do let me know. I’m open for any feedback. I’d love to know how we can make a product that would benefit you.

Agents Bar looking for co-founders

tl;dr: I’m looking for co-founders to help push the Agents Bar to glory. Everyone deserves an easy access to deep reinforcement learning (DRL) agents in their applications.

---

This blog post is one of those very personal ones so brace yourself.

I’m an entrepreneur

There. It’s out. Hopefully, now that’s it’s written and out of my head I won’t have to refer to this again. That word just feels bad. Not only I never had an aspiration to become entrepreneur or a manager, I actively avoided it. Entrepreneurs are obviously people focused only on getting more money. But me, I’m a philosopher/engineer; progress makes me happy. Until quite recently my dream occupation was an ivory-tower academic who, after work, would work a bit more in his own laboratory. Give me physics, medicine and computer science workbooks and leave me alone. Such a vision was based on observing my own progress versus the world progress. Every month I would know much more than I used to know, but the world was changing rather slowly. “Phhh”, my literal thoughts, “give me 20 years and I’m the smartest person alive doing the bestest things possible”. Easy peasy.

Where did I go wrong?

Healthy adult with minimum amount of cynicism can see the flaw. When you have plenty to learn, each attempt moves you further, but pushing boundaries of knowledge is an obstacle course with a lottery at the end. During my World Knowledge Consumption I’ve learn a lot, and those lots include: modesty, realization that none is special, and that it takes many to build something amazing. I want to build something amazing. Initially, I thought that joining amazing people would get me closer to getting amazing stuff out, but that didn’t go well. For one, there just aren’t any amazing people, and two, it’s actually really difficult to help in amazing projects if they don’t let you in. But I do want to do amazing stuff. (Insert here pop-culture meme: Futurama, Bender, casino.) Let’s do amazing stuff.

What’s so amazing about the Agents Bar

I’m still polishing my sales pitch so bare with my rumbling, please. The Agents Bar is a platform, or a bar, where AI agents can hang out. These are (deep) reinforcement learning agents and not your typical chat bots which means that our agents do, rather than talk. If you can describe your circumstances as a state and quantify how good it is to be there, or at least how it compares to a different state, then we provide you an action to take from that state. Common examples are games with some highscore. In fact, recent big news in deep reinforcement learning are about advancements in Atari, Go, Starcraft 2, Dota 2… and many more. But games aren’t the limit. They are simply more natural to apply to (and academics are also humans so they enjoy playing them). You will find more example of reinforcement learning in recommendation engines, in financial advises, inventory optimization, robotics and general decision making. Yes, cheesy as it sounds, reinforcement learning can be applied almost anywhere. One of the reasons why it isn’t, is that it’s difficult to rewrite your problem in terms of state-action, implement the algorithm and run experiments to figure out how to tune the algorithm. It requires plenty of knowledge in many expertise areas and often it’s easier to approximate solution with some heuristic leading to “go enough” and saving on time, or even solving that approximation. However, approximations are always “good enough” or even possible. And that’s where we come in. The Agents Bar solves the problem of implementing algorithms. It solve the problem of needing to run experiments. It doesn’t solve the problem of knowing how to rephrase your problem in state-action space but we’re happy to provide our expertise to help others.

Why now?

Yes, exactly, why not 3 years ago?! We have the technology and it’s a moral sin to leave it as it is. The field of deep reinforcement learning (DRL) is going through a boom. That’s in part of thanks to machine learning (ML) and artificial general intelligence (AGI) hype, and in part thanks to awesome research and impressive PR from DeepMind, Google Brain and OpenAI. There’s plenty of open source implementations, including my own, for DRL agents. Hosting things on the internet are now trivial thanks to cloud providers. Scaling services is easy thanks to Kubernetes and Docker. Big corpos are blaming others for their shitty security which is annoying people and writing plenty of great security tools. Although it is difficult to not include all human disasters when thinking about the World’s state, technologically we are living in the future. That future just needs a bit of moulding.

Why me?

I’ll be honest: there aren’t great reasons why me. As I mentioned, I accepted that I’m nothing special. But, on the other hand, no one is so, heck yes, why not me?! I’ve worked at a huge corp and seen the production of the sausage. I have PhD in mathematical modelling and worked with/on many machine learning techniques. I’m through and through analytical and pragmatic person. Hard working, demanding but definitely fair, and knowing that each person brings their own skill set and we shouldn’t be judged by the same metric. I’m passionate about this, have little obligations (cats and a partner), no life expenses, cheap hobbies. I’m usually good with people as I always try to think from theirs perspective, although my angry-person resting face through some obstacles, especially with western/outgoing people. And, hopefully this is my mic-drop moment, I’ve already built the product. Was hoping to release it today with this post but a few screws got loose so it’s next week.

Impressed?! Come on, BE WITH ME! I’m here for you! Who’s desperate? You’re desperate!

Seriously though

I’m looking for a business partner and/or investors. I always do my homework so I’ve read books and internet facts, and all been taking with hefty doze of logic and rational blend. I don’t want to rush for the sake of rushing and I’m internet says I should be cautious (example links 1, 2, 3). There’s plenty to do and even more not to do. Although I’m motivated I’m still a human and it’s always easier when there’s someone who helps through the adventure. This is not a side project. I’m planning my coming years around this. If this sounds like something you’d be interested in then please do let me know. Feel free to reach out with any questions and/or suggestions. Any feedback is appreciated. (Contact is intentionally not here. It’s easy to find contact to me.)

Parsing TensorBoard data locally

TensorBoard seems to be a de facto standard tool for displaying your ML experimental data. There are some alternatives, but likely due to “being first” and “by google” stamps the TensorBoard (TB) is still the most common one. Many of these alternatives are far superior, especially if you’re using something else than Tensorflow, but TB is still the first go to.

A good thing with TB is that it stores data locally. A bad thing about TB is that it doesn’t provide a nice way to extract that data. (No, clicking one graph at a time and then doing “save as csv” is a terrible way when there are hundreds of graphs to be compared.) To be fair, there is “a way” to convert experimental data into, for example, pandas DataFrames but since it requires uploading and downloading data from someone’s server I’m going to politely ignore the audacity. Ok, it might be the best solution if you have super fast internet connect, a SSD disk, and no RAM memory left on your computer, but otherwise it’s a bazooka shooting a fly.

Do it locally. Tensorboard stores your data in Tensorflow TFEvents records. Tensorflow already has API, summary_iterator, to deal with them. It isn’t magic. Actually, here is a link to my Python gist that does just that – converts TensorBoard data to pandas DataFrame. Note that you need Tensorflow and Pandas installed to use it, which I assume you have since you’re storing as TFevents and want to convert to pandas DataFrame.

Here it is embedded:

	def convert_tb_data(root_dir, sort_by=None):
	"""Convert local TensorBoard data into Pandas DataFrame.
	Function takes the root directory path and recursively parses
	all events data.
	If the `sort_by` value is provided then it will use that column
	to sort values; typically `wall_time` or `step`.
	*Note* that the whole data is converted into a DataFrame.
	Depending on the data size this might take a while. If it takes
	too long then narrow it to some sub-directories.
	Paramters:
	root_dir: (str) path to root dir with tensorboard data.
	sort_by: (optional str) column name to sort by.
	Returns:
	pandas.DataFrame with [wall_time, name, step, value] columns.
	"""
	import os
	import pandas as pd
	from tensorflow.python.summary.summary_iterator import summary_iterator
	def convert_tfevent(filepath):
	return pd.DataFrame([
	parse_tfevent(e) for e in summary_iterator(filepath) if len(e.summary.value)
	])
	def parse_tfevent(tfevent):
	return dict(
	wall_time=tfevent.wall_time,
	name=tfevent.summary.value[0].tag,
	step=tfevent.step,
	value=float(tfevent.summary.value[0].simple_value),
	)
	columns_order = ['wall_time', 'name', 'step', 'value']
	out = []
	for (root, _, filenames) in os.walk(root_dir):
	for filename in filenames:
	if "events.out.tfevents" not in filename:
	continue
	file_full_path = os.path.join(root, filename)
	out.append(convert_tfevent(file_full_path))
	# Concatenate (and sort) all partial individual dataframes
	all_df = pd.concat(out)[columns_order]
	if sort_by is not None:
	all_df = all_df.sort_values(sort_by)
	return all_df.reset_index(drop=True)
	if __name__ == "__main__":
	dir_path = "/home/kretyn/projects/ai-traineree/runs/"
	exp_name = "CartPole-v1_2021-01-26_11:02"
	df = convert_tb_data(f"{dir_path}/{exp_name}")
	print(df.head())

view raw tb2df.py hosted with ❤ by GitHub

Tis the writing season

Usually around this time I spend more days actively thinking and writing. As a boring person I write predominately about ideas for the future, conclusions from the past, experience to repeat, how others should conduct their lives… The boring stuff. Since this is rather cyclical my guess is that it’s mainly because nights are coming early and they bring quietness, bring focus. Coming of the holidays and the new year is also a trigger for reminiscence and planning.

This year there won’t be any holiday planning thanks to the newest startup unicorn that shook disrupted all domains, i.e. the Covid19. There is, however, planning related to the ongoing project le Grand Plan. A quick reminder, as if I wasn’t ostentatious enough about it, I left the job market in order to learn more and potentially contribute to the humanity be that through another company or some research. Everything is going great so far and I really enjoy the self-time. Plans made before leaving are still solid and things are going accordingly. The problem, however, is that assumptions for these plans are falling apart. I no longer “believe” and I’m losing “faith” in their righteousness. (Yes, spiritual references are intentional.) The place where I would see myself in 50 years has shifted. Or, slightly more precisely, the goal is not as rewarding as I thought it would be. Thus the planning.

This is not the note where I share my plans, though I’ve written a lot about my views on the future and might share them in coming months. This note is generally about writing. I used to hate, dislike, avoid, complain, do anything but. Now it’s one of the most important tools in thinking. Since I’m easily distracted, to cope and to be “productive” I needed to find my way of working. Some time ago I shared a template for the project initiation. I still stand by it as being short it allows to initiate projects quickly, keep honest health checks every some often and allows to terminate things that don’t work out. Having projects instead of a thought and crave make things a bit more difficult to give up. Writing out helps. It’s a constantly moving checkpoint. It allows to branch out in thoughts to anywhere and then on dead-end you can clearly see where you were. (Given that I’m currently knees deep in the reinforcement learning I cannot not notice closeness to the Monte Carlo Tree Search‘s Expansion step.) Without scribbling one might forget how they got there and be not able to get back on the original thought. It takes time to learn that the path (narrative) matters more than the exact words or pretty connections. Well, it depends on the purpose, but coming up with great words takes mental effort and that’s taken away from processes that guide in the journey.

One (another?) highlight for future reference is the importance of structure in some planning notes. Since I have many notes in many places sometimes not all of them will be read often. I’ve just found some that I haven’t seen in about 10 years time. Obviously, me in the past wasn’t dumb for thinking such things but they knew little about the world, and if they only knew what I know right now… or, did they know more because they were focused on it, and I just forgot that thing exists? What I want to put in my notes now is a bit more focus on the context. Right now I think that explicitly writing out assumptions, hypothesis, expected results and details of the execution would make learning from the past much easier. It should also make it easier not to tremble in fear after discovering that our ground has been removed weeks ago and we hadn’t fallen because the faith in ground is uplifting. Then the question is whether to safely go down and start building from new foundation, or realise that we’re flying and shoot for the moon?

RuntimeError: cuda runtime error (999) : unknown error at /pytorch/aten/src/THC/THCGeneral.cpp:47

Whenever I’m running PyTorch with enabled CUDA and laptop decides to turn off/put to sleep the GPU, executed script will fail with some CUDA allocation error. Not cool, laptop, super not cool. Restarting the script, or starting any other program that uses CUDA, might not be possible since the memory wasn’t properly released. When trying to run PyTorch you might see the same error as in the title, i.e.

RuntimeError: cuda runtime error (999) : unknown error at /pytorch/aten/src/THC/THCGeneral.cpp:47

Now what? Well, the turn-it-off-and-on™ will work but you don’t have to do all of that. Making sure that the process is properly killed (ha!) and Nvidia’s universal virtual memory (uvm) is clean is enough. The quickest way is to try remove and add nvidia-uvm module with

$ sudo modprob -r nvidia-uvm  # same as sudo rmmod nvidia-uvm
$ sudo modprob nvidia-uvm

Chances are that was enough and you’re good to go. (Note that hyphen “-” is treated equally as the underscore “_” so you might see either version on the internet.)

However, if the remove command above fails with something like “Module nvidia-uvm is in use” then it means that the dead process is alive (double ha!). You will find lsmod and lsof helpful in finding out which modules and processes use nvidia-uvm.

Firstly, try with lsmod by running something like (Note that dot is a wildcard so that it matches

$ lsmod | grep nvidia.uvm

Note that dot "." is a wildcard so that it matches both “-” and “_“. If

kretyn@junk:~$ lsmod | grep nvidia.uvm
nvidia_uvm 966656 2
nvidia 20680704 1206 nvidia_uvm,nvidia_modeset

Columns are “Module”, “Size”, and “Used by”. The number 2 means that there are two modules that use nvidia_uvm although their names aren’t tracked. My observation is that each PyTorch that uses CUDA. To check which modules/processes exactly use the nvidia_uvm you might want to use lsof with something like

$ lsof | grep nvidia.uvm

Note that this might take a while. For me it starts with some docker warnings and then, after ~10 seconds, returns lines with /dev/nvidia-uvm. The second column is what you’re interested as it’s the PID. Take a note of it and check with ps aux whether that’s the process you want to kill, e.g. if PID is 115034 then that’s

$ ps aux | grep 115034  # "115034" is PID and replace with what you want

If that’s proven to be the old, undead then just kill it with kill -9 <PID> and try again with restarting modprod -r nvidia-uvm.

That’s a bit of work when describing but it’s actually quite quick.

Re: Why is machine learning ‘hard’ by S. Zayd Enam

I saw a few people referring to S. Zayd Enam‘s article as an explanation of why machine learning (ML) is hard. It is a good read with some visuals so I’d recommend reading it. However, I don’t entirely agree with author’s arguments and so I thought about highlighting a few nuances.

But, before we go throwing stones, let’s make sure no one gets hurt. Since I’m late to the party (ML discussion parties rock!), my comments are biased from the “now” perspective. I’d like to talk about the craft rather than tools though it’s often not possible as they overlap a lot. Also, Zayd has a disadvantage in that he had published a finite amount of text, thus throwing the anchor, whereas I get to focus on and judge all the details leaving him little space to clarify nuances.

First of all, I agree with the premise that machine learning is “hard”. The hard will mean different things for different people and I might spend the next thousands of words trying to explain what “hard” exactly means but let’s do a short neat trick to speed up everything. Imagine something difficult that would take a day of two to accomplish. The ML is not that; it’s more difficult and it’ll take you longer to accomplish. Ok, sorted.

Regarding nuances in Zayd’s blog post. He compared ML development to programming and not software development. Programming is an essential part of both the machine(!) learning and software development. He somehow acknowledges this by keeping “implementation” and “algorithm” axes in ML problem space. However, with the introduction of the “model” axis means that we’re focusing on a particular subfield of machine learning that has a model. Given that neural approach dominates the ML field that’s likely what most people thought. But in that case we should also focus on some subfields on the other side. How about games development or peer-to-peer network software? I’m guessing that most people will now go either “well this is just silly” or “what’s your point?”. The point is that it might be more obvious that in these particular cases there would be additional axes to think about, for example “performance” and “consistency”. Hold on, but going into details should reduce the problem space, not increase it? Yes. Well, actually, there are many many more axes, infinite to be precise, but because of where we stand and our life experience, most of them are trivial or hidden.

There are two points I’d like to highlight from this “observation”. Firstly, machine learning in its current form is a relatively new field. It’s lacking in tools compared to other popular branches of computer science/application because it hasn’t been widely used for as long as the other ones. The necessity is the driver for development and we can see that there are frameworks building toward easier debugging. That indeed makes ML more difficult right now but it doesn’t mean one being inherently more difficult than the other.
The other point is that most ML practitioners/researchers didn’t study pure ML (what is “pure ML”?) before approaching it. This forces a mental rotation in the problem domain. We have spent some time crafting our approach toolkit but now it seems that those hammers aren’t as useful as before. It’s difficult because the path to the goal, and often the goal itself, haven’t been widely studied.

What Zayd kind of mentioned but didn’t go into details is the complexity. Machine learning is complicated. Same are other branches of computer science/engineering. However, I’ll argue that the ML is an outlier on the complexity axis. An integral part of ML is the input data which don’t even make sense… except that our brain tells us otherwise. All data have noise, be it recording error, or generation process, or quantum wave interaction with the whole universe. This is a problem because we’re trying to understand and utilize some processes but we have little confidence whether our observations are general enough. Extrapolation is hard. Robust methods are hard. Understanding is hard. We’re teaching machines something that we don’t understand ourselves using the same input we were fed with.

All in all, ML is hard. I found it necessary to write out why I don’t agree with the author’s reasoning because understanding the problem is part of the solution. Maybe someone will write why I am wrong, bringing us a step closer to where we want to be?

Leaving Amazon for a sabbatical

It’s been over a week since I left Amazon where I’ve worked for over three years. It seems like a right time to write out some thoughts on the journey.

Disclaimer

First of all, a disclaimer which should be obvious but one never knows. All thoughts are mine and from personal experience. Amazon is a huge company with a total employee counter closer to a million, including a (couple) hundreds thousands software developers. Others have gone through different experiences since they interacted with different people/teams.
Another disclaimer is that I have worked in the corporate side of the company which is different from warehouses and deliveries. Although one of my teams was in transportation and we provided solutions to the yard (roughly “parking outside warehouses”) associates, I still don’t feel confident to talk for that side.

What is good about Amazon?

There are plenty of positive things I would like to say about the company. Off the bat: I would recommend working there. Not everyone, which is why I have left, but I’m guessing it will suit most people. There is a large number of teams to try out and internal relocation policies enable resettling.

A first-time corporate employee, the most impressive thing about software development at Amazon was its internal knowledge. There are plenty of resources to go through and learn. Documentations, videos, design, discussions, internal “stack overflow”… At times it can be challenging to find what you’re looking for but that’s due to ever changing environment and inherited vast amounts of information making the scaling difficult. In addition to the written information, all teams have access to principal engineers or folks who have been for ages, and these are also great to poke their brains.

Another positive is the AWS availability. One can prototype as they wish and try out new features and services. It often simplified my design processes as I could quickly verify whether something works or not instead of getting through layers of documentation. It also removes the burden of constant thinking about costs. When using AWS on my own it matters whether I’m paying $10 or $200 a month but for work projects that’s a prototype cost (threshold is usually agreed with the manager).

Some might also find beneficial the opportunity of wearing many hats. I certainly did enjoy it and it was a great learning experience. SDEs at Amazon are defined by both their scope of influence and their role. You often have to participate in scope projects/product, design solutions, create a prototype, lead the teams, communicate with stakeholder and customers. I’ve seen many opinions on the internet about having many hats as a software developer but I’m of the opinion that one needs to know what and why they are building before actually doing. Developers aren’t “coding monkeys” and they should have a say in whatever they’re constructing. The questions would be more on the balance but, since I’m talking about my experience about Amazon, that balance can be shifted as required.

Why then have I left?

The decision for leaving wasn’t sudden; it was growing in me for over a year. I came to Amazon as a software developer with a PhD and machine learning (ML) experience. I was promised that I’ll be able to utilize my skills in analytical/ML-related challenging projects. That hasn’t happened. In the first year I was in a team with the Away Team model who doesn’t have their own projects. It’s more a mercenary that helps out others with some self-interest. Long story short: others like to keep interesting bits for themselves. The second team felt like a salvation. More promises came but I somehow believed them even though they were far in the future. Then, the future came and it was disappointing. More promises. I was somehow included in and given more analytical projects but they weren’t challenging; the challenge there was managing other’s work rather than working out solutions myself. Higher hopes were related to my latest org, Economical Technologies aka EconTech, which is “reinforcement learning first” org. I was there for about 3 months and it was kind of cosy with great expectations but… everything is just too slow. Not only in the EconTech but the whole Amazon. Taking all my experiences, low faith in any promise, adjusting with the covid-19 expected actions in the, I did a mental forecast for the next year and it has shown no difference. Given no expected progress, deteriorated team inclusively due to the pandemic, slashing my salary due to super-low number of stocks after the fourth year and simple annoyance of Amazon’s stance on increasing global wealth gap while it’s getting richer, well, it’s time to go.

Before going to next thought a quick explanation on what I mean by writing that Amazon is slow. Maybe the analogy to forest fires will be suitable? On the whole it’s quickly spreading and it’s super destructive, but if you focus on a specific point on the circumference then you’ll see that it’s rather slow. It’s slow but it’s a constant steady slow pace. Always that one meter a minute further from the centre. The thing is that the circle at this point is huge. So adding tiny bit in all directions can feel like an exponential growth. Amazon as the company is super fast. There are plenty of new services and ideas each year, and it’s expanding its tentacles almost everywhere. Super impressive! However, if we focus on individual products/teams, then it’s a different story. Most teams are slow. The phrase “it’s always day 1” to me means that everyone is new to the company and they haven’t figured out how to communicate effectively. And there’s new comers syndrome where everyone wants to impress others on their first day which leads to mutually self-imposed high expectations from ill-read peer-pressure. Many will work long, unproductive and mindless hours only decreasing the quality of the product. It’s slow because this only appears as a half-baked product. Is that bad? Well, only if you want to consume that product, otherwise put it on display and it looks awesome.

What am I going to do now?

I’m taking sabbatical for the next 6+ months. In my case, sabbatical means taking the time to focus on the skills I loved to use, i.e. analytical thinking and artificial intelligence. During this time I’d like to catch up with all the advancements in the machine learning world and how this could apply to the current pandemic world. I’m especially interested in focusing more on the (Deep) Reinforcement Learning and creating environments/agents. There are a few thoughts on how I could pay back to society by creating my own product. Probably more on that will come once I have more clarity on the problem and solutions.

Having written that, I’d like to be clear that I’m not closing myself on the outside. I’m happy to listen about all opportunities but I’ll be extremely picky and will prioritise interesting challenges.

And if that doesn’t pay out?

Except for money I’m not losing anything. I don’t need much in life but I acknowledge that I come from a fortunate position. I have everything that I already wanted and there are plans for those that are missing. Everything goes into emergency funds and retirement. If not now, then when?

AI Traineree – PyTorch Deep Reinforcement Learning lib

tl;dr: AiTraineree is a new Deep Reinforcment Learning lib based on PyTorch.

A few months ago by some coincidences at work and some news from newsletters I discovered the world of the Deep Reinforcement Learning. Until now it was “one of those” but on a closer inspection… I couldn’t get my eyes off. Something happened and then it clicked. I’ve started playing around with some gyms from the OpenAI and did a nanodegree course on Udacity, and the feeling grew. So, let me share the feeling.

I’ve started a yet-another Python lib to play around with the Deep Reinforcement Learning. It already has some more popular agents (DQN, PPO, DDPG, MADDPG) and is easy to use with the OpenAI gyms. The intention for the lib is to have a bigger zoo of agents, compatible with more environments and have tools for better developing/debugging. Although it is a work-in-progress project it is already usable. What distinguishes this from many is unification of types and making sure that all components can play nicely with each other. The lib is also based on the PyTorch which I’ve seen many smaller projects with DRL but they usually contain a single agent with a specific environment.

Let me know if you want anything specific in the lib. In a couple of weeks I’m planning to have significant contribution to the lib.

Timed OS light/dark theme switching

tl;dr: A GitHub gist with commands walk-through is available here.

What

The ability to adjust themes, and in particular the dark mode, have been one of the most trendy tech features of 2019/2020. Many sites and apps now allow to to flip between the “normal” and the “dark mode”.

Why

Although I don’t belong to the die hard zealots one can find on the internet, I do appreciate this feature when in dark environment as I’m rather light sensitive and most devices have the lowest brightness on too-high for me. It is a nice surprise that Ubuntu 20.04 came with the global theme and a couple default ones. This let’s me to decide when it’s dark and then switch to the dark mode. Since many pages, e.g. stackoverflow.com or duckduckgo.com, now detect OS’s theme mode they will also be in switch into the mode. Neat. So, when the light goes down, my dark mode goes on, and we’re all happy.

But obviously the night comes everyday so why should I sent those 3 seconds of manual labour when I can make it automatic?

How

I won’t into too much details but basically the proposed solution is using a service manager systemd and more specifically its systemctl command. There are two “services”, one for each theme flip, and they are run daily at specific time.

For the servicd to automagically detect your services and timers they can be be placed in ~/.config/systemd/user. It’s likely that there isn’t such directory so create it. The code also expects that there is a directory ~/.scripts where some random utility scripts are placed.

The walkthrough code is below. Please note that none of the files are expected to be where they are so you have to create them and fill with the content that the cat command returned. Also, the script changes default terminal profile and it’s expectating that there are two called “Dark” and “Light” for day and night, respectively.

user@host:~/$ mkdir -p ~/.config/systemd/user
user@host:~/$ mkdir -p ~/.scripts

user@host:~/$ cat ~/.config/systemd/user/dark.service  # Create this file
[Unit]
Description=Automatically change the "Window Theme" to "light" in the morning.

[Service]
Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
ExecStart=~/.scripts/profile_changer.sh light

user@host:~/$ cat .config/systemd/user/light.timer
[Unit]
Description=Automatically change the "Window Theme" to "light" in the morning.

[Timer]
OnCalendar=*-*-* 06:00:00
Persistent=true

[Install]
WantedBy=default.target

user@host:~/$ cat .config/systemd/userdark.service  # Create this file
[Unit]
Description=Automatically change the "Window Theme" to "dark" in the evening.

[Service]
Environment=DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
ExecStart=~/.scripts/profile_changer.sh dark

user@host:~/$ cat .config/systemd/user/dark.timer  # Create this file
[Unit]
Description=Automatically change the "Window Theme" to "dark" in the evening.

[Timer]
OnCalendar=*-*-* 19:00:00
Persistent=true

[Install]
WantedBy=default.target

user@host:~/$ cat .scripts/profile_changer.sh  # Create this file
#!/bin/bash

get_uuid() {
  # Print the UUID linked to the profile name sent in parameter
  local profile_name=$1
  profiles=($(gsettings get org.gnome.Terminal.ProfilesList list | tr -d "[]\',"))
  for i in ${!profiles[*]}
    do
      local uuid="$(dconf read /org/gnome/terminal/legacy/profiles:/:${profiles[i]}/visible-name)"
      if [[ "${uuid,,}" = "'${profile_name,,}'" ]]
        then echo "${profiles[i]}"
        return 0
      fi
  done
  echo "$profile_name"
}

if [ $1 == "dark" ]; then
  THEME='Yaru-dark'
elif [ $1 == "light" ]; then
  THEME='Yaru-light'
fi
UUID=$(get_uuid $1)

/usr/bin/gsettings set org.gnome.desktop.interface gtk-theme $THEME
/usr/bin/gsettings set org.gnome.Terminal.ProfilesList default $UUID

user@host:~$ chmod a+x .scripts/profile_changer.sh  # Make script executable
user@host:~$ systemctl --user daemon-reload
user@host:~$ systemctl --user enable dark.timer light.timer
user@host:~$ systemctl --user start dark.timer light.timer

The last three commands will refresh the service daemon to and make it look for any file changes, enable timer services to be run in the background on startup and start them now.

That’s less work than expected initially. As most of the time, most of the work came from the StackExchange, from the AskUbuntu thread. Lucky that most of the time there’s someone with a similar question and someone with good answer.