Category: Startups

Agile for Start-Ups – part 1

October 28, 2021 /

I want to start these two articles by observing that I don’t believe agile methods are appropriate in their proper form for Start-Ups. In the first article, I will list most of the disadvantages of the agile methodologies and how they could hurt the Start-Up’s performance. In the second one, I shall present what worked for me in the past and what did not. 

But let’s start with little history of the product management frameworks and how they evolved. Historically the initial management framework was the so-named waterfall or phased approach. With that approach, the companies copied how most engineering products are created by multiple phases and initial requirements gathering. This way of working does an excellent job when you want your solution to be installed on-premise and must work for a long time without interruption. 

In the 80s of the last century, Watts Humphrey, during his work at Carnegie Mellon University, founded the Software Process Program, and with that, he started the so-named PSP (Personal Software Process). This process heavily relies on not comparing the performance between different developers but rather trying to improve every developer as a single unit. Additionally, in PSP, we have the notation of knowledge re-usage – aka if the developer has implemented something similar as a block in the past, we could expect him to implement such block with the same speed.

In 1986 Hirotaka Takeuchi and Ikujiro Nonaka started Scrum and the foundation of different agile methodologies. At its core, the agile development process defines that the “customer/client/company owner” will work together with the team and provide business knowledge and requirements weekly. A fundamental principle of Scrum is the dual recognition that customers will change the scope of what is wanted (often called requirements volatility) and that there will be unpredictable challenges — for which a predictive or planned approach is not suited. These changes come from various sources, but understanding why is irrelevant: change should be accepted, embraced, and analyzed for benefits.

On the diagram, you can see the standard agile workflow. Its typical duration is between 1 and 4 weeks. In Start-Ups, the main problem is during the planning meeting because it is impossible to plan R&D

And now here are some reasons they do not work well for complex Start-Ups:

  • R&D: All of the listed methods work on the assumption that the team will not do any research and development. Aka, the business owners know their business niche and that someone else already did the R&D part. In reality, this rarely happens, except when we start a business without any innovation. When you want to make an innovation, all traditional product management frameworks stop working. Usually, in the Start-Up World, the team must do R&D and code writing simultaneously (investors and clients want to see progress).
  • Burnouts: Constant changes in business requirements and business situations usually involve many changes in the technology stack and your programming code. Sometimes you must rewrite the whole product from zero. And this typically takes its toll on the engineering team. For sure, if you do not manage to make business traction (aka finding clients) for the first three years of your Start-Up, it is virtually certain that this lack of progress will demotivate your technical team, and most probably some of the members will feel the effects of burnout.
  • The process will not mitigate your people’s weaknesses: According to most agile methodologies fans, these methods are the panacea for every technological problem. However, we have to ask ourselves why only 5% of the Start-Up technical companies pass the second year of work? All of them use these methodologies. For sure, we could blame the business development and marketing, and we will be right that 70-80% of one successful business is the way you sell. However, at the same time, I saw many technical teams struggling in their performance because of leaning too much on the agile principles.

In conclusion, I would want to emphasize that no product management workflow in the World will mitigate the weaknesses your team has in the first three years of your Start-Up. The only way to increase your team productivity is by training them, and by training, I mean the business and the psychological part of being into a Start-Up. Technical knowledge is much easier to adopt and realize than handling that your odds of winning a VC are 1:300.

Could Open Source be successful for Start-Ups

October 21, 2021 /

For a long time, I had this discussion with colleagues and friends about whether open source technologies, especially Linux, could replace all of your software needs. In the past, the main problem regarding using Linux was not mature enough ecosystem and being too complex to work with. However, after the introduction of Android (we could categorize it as a Linux distribution), things became less difficult, and many companies invested much of their time into making their products work on Linux. 

But still, something is missing. By official data, the Desktop Linux users are between 0.6 and 1.5% of the overall Desktop Users. And this is quite a low count of Desktop users. It seems free is not enough for business owners to make the shift. But let’s compare the traditional business software stack to what Linux can offer:

A typical modern office setup, in which there is a data center with servers, and all of the employees are using these servers to connect to their corporate network and use the systems
  • Microsoft Office and Outlook: LibreOffice is an excellent alternative for having an on-premise installation of the office software package. It usually has good compatibility with the original Microsoft Office and could open and edit files. Sometimes the styles of the original files are destroyed. In that case, one can use the cloud variant of Microsoft Office in a web browser. This way, we could avoid such issues. Additionally, Linux comes with Thunderbird, which is an excellent alternative to Outlook.
  • Zoom/Viber/Microsoft Teams/Slack: All of these have binaries for Linux, which means you could have the standard video conferencing apps installed on your machine and have the same experience as the Windows-native users. Additionally, they all support web browsers, which means no need for a native app for communication.
  • Exchange/Sharepoint: Most Linux distributions do not support an active directory out of the box. However, one German distro supported all the group policy features and made it possible for Ubuntu-based clients to connect to this active directory. The name of that distribution is Univention Corporate Server and could be used as a drop-in replacement for Windows Server.
  • Specialized Software: And usually, here comes the main problem with Linux. Most of the specialized software does not have builds for Linux. Some examples are Adobe Photoshop, Adobe Illustrator, 3D rendering software, most of the accounting software, most of the governments’ software, etc. Unfortunately, there are no signs of bringing this software to Linux soon. Fortunately, most of the workers in a given company do not need highly specialized software, and they could do their job in the web browser.

In conclusion, using Linux in the corporate environment has become more and more user-friendly. At the same time, I firmly believe that Linux could entirely replace the software stack for SMEs, excluding those using the specialized software. Fortunately, there is a shift in software development for going into the cloud, which could, even more, help the SMEs with the specialized software (some vendors are already moving their software in the cloud). Additionally, all Linux distributions support Firefox and Chrome out of the box. And as a final – during my coronavirus sick leave, I managed to open my X-Ray photos (DCM image format) on CentOS 8 (I have been using CentOS for the last up to 10 years) without installing anything. ImageMagick supported it out of the box.

Real time body camera system – Camera Device – part 2

October 14, 2021 /

In the last part, we finished the description of our network protocol and its advantages over other encrypted video streaming protocols. In this part, we shall discuss how we created our hardware prototype for the body camera system and what performance problems we had to resolve when we implemented the software part of it. At the end of the article, we shall show you how much our prototype costs and a sample budget for doing something similar.

But before that, let’s first see what our competition was and what features they had for their cameras.

Axon Body 2

The Axon Body 2 is a camera system incorporating an audio and video recording device. This camera is designed for use in harsh environmental conditions encountered in law enforcement, corrections, military, and security activities. The Axon Body 2 camera is designed to record events for secure storage, retrieval, and analysis via Evidence.com services. The recorded events are transferred to your storage solution via the Axon Dock or by using Evidence Sync software installed on a Windows computer.

  • HD Video and Dual Audio Channels: Record in low-light and HD, and make voices more distinct with automatic tuning and noise reduction.
  • Wireless Activation: Axon Signal reports events, like when you open the car door or activate the light bar, so your camera can start recording.
  • Wi-Fi & Bluetooth Connectivity: Use Wi-Fi to stream videos and Bluetooth to assign metadata.
  • Mobile App: Connect with Axon View to stream, tag, and replay videos from your phone.
  • Unmatched Durability: Handle in extreme weather and brutal conditions.
  • Full-Shift Battery: Record for more than 12 hours.
  • Axon RapidLock Mounts: Keep your shot steady with versatile mounts.

Motorola V300 Body Camera

This camera is built specifically for law enforcement. The V300 continuous-operation body-worn camera is ready to go when you are with its detachable battery, 128GB of storage space, wireless uploading, and Record-after-the-Fact® technology. Integrated with the technology you use daily to enhance your focus and combined with powerful device and evidence management software, the V300 body-worn video solution enables you to capture every encounter. 

  • Detachable Battery: Easily change the V300’s rechargeable battery while on the go. Keep an extra battery at the ready for unexpectedly long shifts, extra shifts, or part-time jobs where a body-worn camera is required.
  • Natural Field of View: Eliminate the fisheye effect from wide-angle lenses that warps video footage. Our distortion-correction technology provides clear and complete video evidence.
  • Built-in Display: A clear LCD on the top of the camera allows easy viewing of device status.
  • Absolute Encryption: Elevate your data security with encryption at rest and in transit. The V300 guards your data and your reputation.
  • Rugged & Durable: Tested ruthlessly to survive in a public safety environment, the V300 is shockproof and waterproof to IP67.
  • Automatic Wireless Upload: Send critical video back to headquarters while still in the field. When docked in the car, the V300 body camera uploads to cloud-based or on-premise evidence management systems via wireless networks like LTE and FirstNet, anytime, anywhere.

During the time of development, these were the two main competitions. Both of them lacked the real-time streaming support we wanted. However, both of them had pretty exciting features, without which our solution would not have enough commercial traction. 

After a good amount of market analysis and tests of different technologies, we decided our body camera system to have the following features:

  • Full-Shift Battery: Record for more than 12 hours.
  • Automatic Upload: Send critical video back to headquarters while still in the field.
  • LTE Real-Time Streaming: With adaptive bitrate, we could make our camera system send data during the whole shift.
  • Rugged & Durable: Tested ruthlessly to survive in a public safety environment
  • Built-in Display: A clear LCD in the camera system to allow easy viewing of system status.
  • Absolute Encryption: We wanted data security with encryption at rest and in transit.
  • Fisheye Field Of View: We wanted our camera system to support more than 100 degrees field of view.
  • Low Light Vision: Having in mind that most of the crimes happen during the night, we wanted this feature.

But we had a problem. Being a small, underfunded team located in Burgas, we did not have access to many hardware vendors, nor did we have the hardware team who could implement a body camera system from scratch. We had to take another approach. After a couple of weeks of analysis, we decided to implement a pluggable system using manufactured customer devices. The final system design consisted of the following components:

Hardware

  • Android-based hardware device: For the last decade, almost all Android devices have supported USB On-The-Go. USB On-The-Go (USB OTG or just OTG) is a specification first used in late 2001 that allows USB devices, such as tablets or smartphones, to act as a host, allowing other USB devices, such as USB flash drives, digital cameras, mouse or keyboards, to be attached to them. USB OTG allows those devices to switch back and forth between the roles of Host and Device. A mobile phone may read from removable media as the Host but present itself as a (USB Mass Storage) Device when connected to a host computer. In short, we could attach a standard USB web camera to a typical smartphone.
  • Body mounted USB camera: Here, we had quite an interesting problem. Standard USB web cameras are not tailored for body mounting, neither are they durable enough. We spent a good amount of time checking how to solve this issue, and finally, we managed to find a suitable USB camera vendor using Sony-based camera sensors. The vendor could mount any lens to the camera sensor, and the whole board came with a good amount of mounting holes. After that, one of our hardware people designed a custom mountable case for our USB camera and 3d printed it.
  • New extended battery: The standard battery of our Android device was around 4100mah. Unfortunately, after multiple tests, we saw that with every needed hardware capability activated, aka LTE, USB OTG, GPS, and microphone, the Android device was taking around 800-900mah per hour. And this was not enough for the whole 12 hours shift. So we took the extraordinary decision of creating our battery. Finally, we managed to produce a proof of concept 12400 mah battery replacement for our Android device. And indeed, it took 12 hours to recharge.
  •  Mount for cars and bicycles: We wanted our system to support multiple different mounting points. So, to allow this to happen, we bought standard multi-camera mounts for vehicles and bikes and created adapters for our 3d printed camera to enable attachment to the stock mounts. 

Software

On the diagram, you can see a sample architecture diagram of the solution. With that architecture, we managed to achieve 22 frames per second with streaming and encryption.
  • UDP streamer module: This module’s main functionality was sending UDP packets and receiving answers for these UDP packets. It sent analytics data to the Adaptive bitrate control module to decide how to switch between different formats and resolutions.
  • Encryption module: This module was highly optimized to perform hybrid encryption and decryption of byte objects. We managed to optimize the performance, so the module supported encryption and decryption of real-time h.264 frames coming from the USB module.
  • Network protocol module: Main functionality here was to construct and decode UDP datagrams messages. It used the encryption module to encrypt the data before sending it to the UDP streamer.
  • Adaptive bitrate and codec control module: This module controlled what type of compression strategy to use to ensure that the headquarters will receive data no matter the LTE signal. 
  • Objects pool module: The idea of the module was to reuse different bytes arrays during the lifecycle of the h.264 packets. With around 24 frames streamed per second, creating and destroying many bytes arrays would entirely kill our application.
  • USB camera module: This module wrapped the communication and handling of the USB video camera bus. The idea was to support multiple different cameras and formats here.
  • Telemetry module: In this module, we collected all the additional data we had – current battery consumption, remaining battery time, GPS coordinates, sd card storage, etc.
  • h.264 decoding module: This module’s main functionality was to transfer video frame data in a different format. For example, we supported h.264 frames, png, and jpeg formats. The application was intelligent enough to decide when to switch between the different formats.

We used Java and C++ programming languages for the implementation of all the modules. The only C++ part was the USB camera module because of the low-level communication with the USB bus. 

Let me share some notes on why we decided to use an Android device. We could implement our body camera system using an ARM-based board with Linux installed on top of it. It would dramatically reduce our software efforts. However, from a hardware point of view, most ARM-based boards lacked good CPUs, battery support, and housing. Not to mention, the development of a custom ARM board was entirely outside of our budget. Fortunately, our software was designed this way, so we could easily switch the hardware platform in case of investment or more considerable client interest.

In conclusion, our body camera system managed to fulfill our initial requirements for MVP. It worked well, and we made multiple videos and streams testing it in various environments and locations. Our system even managed to send data through 3G mobile cells in areas where LTE/4G was not supported.

A sample video of how the system works could be found here

Some photos of our POC build. With that build we managed to fulfill our requirements.

Start-Ups: Endgame

October 14, 2021 /

After the two weeks-long enforced COVID pause I had to endure, now I am back with the last part of the StartUp Lifecycle series. In this part, we shall speak about what happens after the Start-Up has multiple rounds of bank loans and the usual working strategy after the Start-Up has reached the IPO. Previous parts of the series you can find here and here.

Now, once the Start-Up reaches the IPO status, the modus operandi has to change a bit. The company has to prove itself as a leader in the field and acquire as many clients as possible. Usually, the founders try to balance between too many bank loans and enough income to pay for the developed infrastructure and employees. At that phase, the Start-Up is no longer “categorized” as a Start-Up but usually as a mature and more giant company. Better and more mature processes are established, and usually, the management has to find a way to delegate and distribute the management power and duties.

On the diagram, you can see the standard lifecycle of the given company after IPO. A merger is the most common exit these days.

On the economic side of things, we could not expect new investments and fundings. Usually, the board of directors is trying to survive on IPO and profits. As a rule of thumb, we could expect the company to operate at a loss and cover this loss using IPO profits or bank loans. This way of operation usually gives a good long run of business execution. With this strategy, the company can survive for around 10-15 years, and during this lifespan, the company owners have three options:

  • To find another company for merger or acquisition: At this stage, the company usually has enough assets and IP, which could interest another company. Mergers and acquisitions are typically categorized as successful exits and will leave founders’ reputations intact.
  • To make the company run on profit: Some owners could decide to stop the company’s growth and focus on getting enough clients to keep the company on profit. That was not a rare choice in the past. However, many company owners will pursue option one because it gives them less risk in the long term.
  • To fill bankrupt: As everything in our world, companies could come to an end. Balancing between shares, bank loans, and profit could be tricky sometimes and lead to erroneous results. Significantly, the share price is sometimes quite volatile and could be affected by the CEO’s matters and life choices.

In conclusion, at that stage, companies rarely fall bankrupt. Most owners and major shareholders would prefer to sell the company and its assets instead of bankrupting. At least this way, the employees usually retain their jobs and can be moved to more successful projects in the new company/structure.

Game of loans

September 8, 2021 /

In my article on start-up unicorns, I already presented how most start-ups finance their operations and how efficient this way of work is. This article will show how companies and wealthy individuals finance their operations once they reach unicorn status and have already managed to execute a successful IPO or ICO.

But before explaining the financial workflow, let’s analyze what an IPO is and how it integrates with the standard capitalism-based system. At its core, IPO operates the same way as every ordinary bank. People trust the company doing IPO and are willing to buy common stocks of this company. Additionally, let’s analyze a little bit how banks evaluate a given company to calculate its value. Usually, it is a combination of all of its assets, including the common stocks from the stock exchange. So far, so good; however, the stock exchange evaluation rules are pretty exciting. More specifically, the rule of how the end-of-day price calculation is done. It is based on the amount of money an individual is willing to pay for a given stock. And here is the part that must bother us – one big chunk of a given company evaluation is entirely based on people’s trust in the company. It is not based on any real-life assets such as gold, art, or real estate. It is entirely based on faith. We can even safely assume that we ll are living in a trust-based economy.

On the diagram, you can see a standard way of how wealthy individuals finance their operations. They use the IPO/ICO to increase their liquidity and apply for a loan after that

But let’s go back to loans – how do we calculate the personal wealth of people. The answer is a simple one. The same way banks calculate the evaluation of a company – aka based on all personal assets, including stocks. When wealthy individuals decide to buy something or invest in something, they have two ways of doing that – to sell assets or to get a loan.

Usually, most of them are willing to get a loan based on the current evaluation of their stocks and payback later. However, to give the loan, the bank does the review based on the willingness of someone to buy the stocks at a given price. In traditional banking, this usually triggers the central banks to issue a new amount of the local currency to provide the bank with the amount of money necessary to give the loan. So, in short, every time the bank provides a loan based on stocks, we pump new money into the system and lower down the buying power of everyone attached to the local currency.

In conclusion, most wealthy individuals prefer to finance their operations using loans instead of selling stocks at the current value. However, getting a loan increases inflation because the central banks have to issue new money to fund these loans. Another question is how much is the buying power of our modern billionaires compared to the ones in the past. For sure, most of them can not afford to finance the operations of over 1000 public libraries with their own money.

Five mistakes to avoid when building your startup

September 2, 2021 /

For almost 18 years, I have been working in product-based Start-Ups. During this time, I have seen a fantastic range of mistakes made during virtually every stage of their lifecycle. This range starts with something small, such as wrong employees’ computer equipment and massive investments in expensive server equipment or shady marketing agencies. However, I can categorize five mistakes as showstoppers for every Start-Up. They can instantly kill your company:

  • No business need: Unfortunately, many companies start developing a product without proper business research. I have done that at least five times in my professional journey. However, creating a technical product without adequate business verification is the number one reason for a Start-Up failure.
  • Erroneous business team: As we speak about business, many entrepreneurs and investors follow the “A player” hiring mantra. The business development teams in the Start-Ups I was part of were with quite mixed backgrounds (including people from Harvard and St. Gallen). And still, the results were mixed. You will need the team, which can do the job for you, but not the team with the flashiest CVs.
On the diagram, you can see a standard distribution of the mistakes made in one start-up. The most significant percentage is always for no business need
  • Erroneous technical team: Absolutely the same as the previous point, but for your technical team structure. I have worked with people from different backgrounds (including people from companies such as Google, Facebook, Twitter, Amazon, etc.). Again the results were quite mixed. I cannot deduce a trend where the more prominent the background is, the better. The only trend I could figure was that your team needs the right attitude.
  • Not enough team compensation: Many entrepreneurs think they must take a big part of the equity pie after giving the money and the idea. However, this kind of thinking is wrong. Ideas and money are nothing without proper execution. And if you cannot motivate your team to execute, this is quite an excellent way to shoot yourself in the foot.
  • Aiming too high: Many Start-Ups aim too high in terms of customers. However, this is quite a harmful strategy, bearing in mind that big companies’ decision-making process is notoriously slow. Better start small and acquire a pool of smaller customers and then scale (ideally, you can bootstrap this part and take funding only for marketing and scaling). Using this strategy, you achieve two things – traction and early verification. Hunting deers and elephants[1][2] can come on the next iteration.

In conclusion, building Start-Ups is hard. Almost 95%[3][4] of the Start-Ups fail during the first 2-3 years of their lifeline. Keeping in mind that people around the World start over 100M new Start-Ups every year, this is a sad statement. The listed five mistakes and given that the average Start-Up founder comes with a huge ego are a recipe for trouble. Leaving you with a thought  – you can print new money, but you cannot issue new brains. Please treat your team well.

[1] – https://www.slideshare.net/theproductguy/elephants-deer-rabbits-choosing-the-right-customer-for-your-products

[2] – https://kimtasso.com/selling-basics-targeting-with-rabbits-deer-and-elephants-video/

[3] – https://www.investopedia.com/articles/personal-finance/040915/how-many-startups-fail-and-why.asp

[4] – https://medium.com/journal-of-empirical-entrepreneurship/dissecting-startup-failure-by-stage-34bb70354a36

We live in а bubble world, full of unicorns

August 12, 2021 /

If we can use one sentence regarding the startup culture for the past decade, it will be Bubbles and Unicorns everywhere. These days every entrepreneur is trying to create the next unicorn and to fill its bubble with money. Unfortunately, looking at the facts, this approach is unsuccessful and usually leads to the startup’s failure. 

But let’s analyze the definition of the unicorn in the startup culture – the unicorn is a startup with an evaluation of over 1 billion US dollars. Here is the essential point of our analysis. Evaluation is not the annual profit these companies are making. We could define evaluation as the “social” trust into a global brand, and we “evaluate” this “social” trust to 1 billion US dollars. Last year’s report showed that from 73 unicorn companies, only 6 had a positive net profit. And to make the situation even worse, 34 of these 73 unicorns had losses more significant than 30% of their revenue.

Having the previous paragraph in mind, we could easily deduce that without a proper IPO, these 34 unicorns would most probably end with a failure. Furthermore, the six profitable Unicorn startups (out of 73) did IPOs many years ago. No Unicorn startup among those announcing or doing an IPO since Zoom in August 2019 was profitable in 2019 (or 2020). The statistic suggests that the privately held Unicorns, which have yet to do IPOs, are primarily unprofitable. Thus, the record low profitability of startups is likely to get worse. 

On the diagram, you can see a standard workflow of the making of a unicorn startup. In every round, more and more companies decide to exit, or they fail. Others manage to attract new funding and continue until they reach the dreamt status of the unicorn

And here is one exciting statement – every IPO company acts as an investment bank. It needs the IPO investors’ money to fund its activities. However, we need to ask ourselves whether this is a sustainable approach and whether we should mark unicorns as “successful” business ventures, considering that they do not operate on profit. And here is a sample list of reasons, marking unicorn as successful is a bad idea:

  • No net profit from their main business idea: Making not enough profit from their business idea means that the business idea is not viable. Evaluation of 1 billion US dollars does not mean that the assessment of the concept is so much.
  • Need of IPO to survive: Going into IPO mode means that the unicorn is now a public investment bank. In that sense, it starts acting as a bank, but not as a business venture. Another hint that the business idea was not profitable enough.
  • The considerable margin between evaluation and revenue: Sometimes, there is a significant margin between profit and revenue. It is essential to understand that evaluation is based on mathematical formulas, and as with every formula, the results can be set up for enormous evaluation. So, in short, a significant evaluation does not mean a successful business idea.

In conclusion, I think it is suitable for every entrepreneur and investor to ask themselves whether it is good to operate a business this way. Maybe a more prudent approach to doing business and ensuring the company has a minimized positive net profit will make the startup environment much better and less stressful.

Why startups experience is like being in the SAS (Special Air Service)?

August 5, 2021 /

Authors around the World publish tons of books on how to create your startup and how to scale it to a multi-billion company. People read these books, praise them and try mimicking the strategies written there. Even courses train you to present yourself in front of investors and get the next significant investment for your startup. All of these books give hope and motivation to the current and future generation of entrepreneurs.

Still, year after year, we see the same trend – 90% of the starting companies will fail until their 2nd year. Or in more human-readable wording – 90% of the new companies can not reach the sustainable revenue phase, and their bubble burst until their 2nd year. At the same time, 97% of the latest companies fail until their 5th year. This statistics is quite sad because it shows that all the courses and books on the World are not enough for your startup to succeed. You need experience and first point of view knowledge of how things are working and what is necessary for success.

On the diagram, you can see a standard corporation versus startup skills distribution. Startups team members need to understand the business side much more than the regular corporation employee

Many people do not realize how difficult it is to create a startup. It would help if you had lots of experience to make it happen. 99% percent of the population on our planet do not have this experience, and to gain it, they need to fail. And to fail hard and often. Let’s analyze why 90% of the startups fail until their second year of running.

  • The average length of an IT project is between 18 and 24 months. If you do not manage to scale your product for this period, then, most probably, your business model does not work, and it will not scale at all.
  • The average person has some resources put aside for this period. If you are trying to make a bootstrapped business, this period is your lifeline to achieving any progress.
  • In case you manage to gain traction for your startup idea. Many people do not know how to scale it out and make this traction a sustainable business. One of the biggest problems is customer support after you manage to get the initial traction.
  • Let’s analyze the stats about startups’ failure. 90% of the startups fail until the second year. It directly says – you will need, on average, nine failures to pass the second year of startup life. If we multiply this number to 18 months (average lifetime of one IT project), then we receive 9 * 18 = 162 months or almost 14 years of working in startups to make one of them successful. That’s why most of the time, one startup needs at least two or three co-founders with enough experience in startups to scale.

In conclusion, making a startup is hard. It is not for everyone, and many people lost time and money trying to create one. Without the proper experience and coaching, the failure of startups will continue. From my personal experience working in startups, some of them, relatively underfunded; if you pass the second year, your chances of success improve dramatically. And yes, the SAS drop rate is, on average, around 94%.

Cybersecurity tactics for small teams – Network Security – part 2

August 1, 2021 /

Please check the previous part – here.

Now, as we can see, attackers can penetrate all of the hardware network devices we reviewed. How easily it depends on how do you set up your cybersecurity and patch policy. 

It is clear that despite your best effort, you must not blindly trust your routers and switches. Lack of trust is precisely the paradigm behind the zero trust model. At the same time, to make attackers’ life harder, it is important to mention three types of defensive cybersecurity tools, which can help you increase your defenses and trust in your local network. 

Until the end of this article, I shall describe them. As a final, I shall give a sample budget for both router and switch devices. Both of them will use open-source software. Usually, they receive software updates quite often and can offer your a good level of security.

Firewalls

A firewall is a network security service that monitors incoming and outgoing network traffic and decides whether to allow or block specific traffic based on a defined set of security rules.

Firewalls have been the first line of defense in network security for over 25 years. They establish a barrier between secured and controlled internal networks that you can trust and untrusted outside networks, such as the Internet. 

Usually, in the case of home networks, this service is deployed in your hardware router. The last sentence means that the attacker will expose your entire local network to the Internet in case of router penetration.

Intrusion Detection/Prevention

An intrusion prevention system (IPS) is a form of network security that detects and prevents identified threats. Intrusion prevention systems continuously monitor your network, looking for possible malicious incidents and capturing information about them. The IPS reports these events to system administrators and takes preventative action, such as closing access points and configuring firewalls to prevent future attacks.

An intrusion detection system (IDS) is a device or software application that monitors a network or systems for malicious activity or policy violations. Any intrusion activity or breach is typically reported either to an administrator or collected centrally using a security information and event management (SIEM) system. A SIEM system combines outputs from multiple sources and uses alarm filtering techniques to distinguish malicious activity from false alarms.

At home routers, intrusion prevention systems can be deployed on the router device and inspect all the incoming network packets from the Internet. On the other hand, an intrusion detection system is deployed on all the hardware devices connected to your local network. In simpler words, prevention systems monitor your incoming traffic, and detection systems monitor your local network for anomalies. 

On the diagram, you can see a standard SIEM system. The idea of the system is to aggregate all of your logs and data and offer analytics to your security engineers

Group Policy

Group Policies, in part, control what users can and cannot do on a computer system. For example, a Group Policy can enforce a password complexity policy that prevents users from choosing an overly simple password. Other examples include allowing or preventing unidentified users from remote computers to connect to a network share or block/restrict specific folders. A set of such configurations is called a Group Policy Object (GPO). 

Now, group policies can be a powerful instrument for system administrators to define how the organization computers act to different security threats. Unfortunately, Group Policy settings are enforced voluntarily by the targeted applications. In many cases, this merely consists of disabling the user interface for a particular function of accessing it. Alternatively, a malicious user can modify or interfere with the application to not successfully read its Group Policy settings, thus enforcing potentially lower security defaults or even returning arbitrary values.

For home-based local networks, the usage of group policies is usually limited. Still, I would advise system administrators of small teams to think carefully, how to add group policies to their remote office deployments. In combination with VPN, Group Policies can add much value to your cybersecurity workflow.

Budget

As I promised at the beginning of this series, I shall give a sample security budget for every topic we discuss. Again I will tailor the budget to small teams with an underfunded budget for cybersecurity defenses. 

  • Router (180$): I would go for Pfsense or IPFire based hardware appliances. Both provide reasonable protections, even though the first is based on FreeBSD and the second is a Linux distribution. Both have state-of-the-art firewalls, and both support Snort and Suricata, which are the best intrusion prevention systems. Additionally, they have Syslog support so that the router can become a part of an intrusion detection system.
  • Switch (150$): SwitchBlox Rugged is a good option here. It is a little bit more expensive than the standard network switches. However, it comes with an open-source networking operating system and can work in harsh environments. Two switches can be stacked together.
  • SIEM System (0$): MozDef is a SIEM system developed by Mozilla. It is open-source and supports all the necessary features for SIEM systems. 
  • Group Policy Server (150$): We can order PC Engine’s apu4d4 unit and install on top of it Univention Corporate Server. With it, we can create policies for Ubuntu and Windows-based computers.

With a total budget of around 480$, we achieved a pretty good level of security. Still, a determined attacker can penetrate this setup, but it will take him more time and resources. The switch is optional, but it will help if you want improved security and choose to have a WiFi network for guests only.

In conclusion, setting up a network perimeter can be done effectively on a budget. Still, it is essential to note that the budget does not include the human hours needed to set up the equipment and your local network.

Next part is – here.

Real time body camera system – Network Protocol – part 1

July 1, 2021 /

In this series of articles, we shall discuss one of my old projects. During that time, I had a consulting company working in IT, and this project was part of my initial steps in cybersecurity. The project started around the middle of 2015 and ceased to exist at the end of 2016. It is in body cameras, and actually, it was a competition to systems such as Axon Body 3 camera. During the lifecycle of this project, Axon cameras did not support LTE-based streaming.

The team around the project and I managed to produce a working prototype of the system, and in this series, I shall present to you how we implemented the prototype. At the end of the articles, I shall show you the actual budget for doing this prototype and analyze why it was unsuccessful. 

The topic of this part will be an analysis of the advantages and disadvantages of the current video streaming network protocols. We shall start with the standard video streaming protocols, and at the end of the article, we shall discuss our modified, more secure protocol.

There are multiple different protocols for video streaming. Part of them do not support encryption, and we shall focus ourselves on those which support it.

RTMPe

Real-Time Messaging Protocol or RTMP is used to stream multimedia data – audio and video – between Flash Media Server and Flash Player. The chief utility of the RTMP stream is in the optimization of the audio and video data transfer between the server and player.

Encrypted RTMP (RTMPE) wraps the RTMP stream session in a lightweight encryption layer. Through Encrypted RTMPE, the streaming protocol provides low-level stream encryptions for high-traffic sites. RTMPE uses the Anonymous Diffie-Hellman key exchange method. In this algorithm, two parties – the media server and the flash player – establish a shared secret key over an insecure channel.

The standard RMTP protocol uses TCP, and RTPMe uses an encryption model based on a shared secret.

HTTP Live Streaming Encryption Methods

While the HLS supports AES-128 encryption, there are two different ways to implement the standard in practice.

Broadcasters can use one key to encrypt the entire video stream, but that also means the whole stream is unprotected if an unauthorized third party intercepts the secret key.

Alternatively, each segment of a stream can be encrypted with a different key. That way, users can access only a few seconds of video with each specific key. Broadcasters might choose this method if the video content their sharing is highly sensitive.

As it comes from its name, HTTP Streaming uses HTTP to resemble MPEG-DASH. It works by breaking the overall stream into a sequence of small HTTP-based file downloads, each downloading one short chunk of a broad, potentially unbounded transport stream. A list of available streams, encoded at different bit rates, is sent to the client using an extended M3U playlist. HTTP is a TCP-based protocol, as well.

MPEG DASH Encryption

Dynamic Adaptive Streaming over HTTP (DASH), also known as MPEG-DASH, is an adaptive bitrate streaming technique that enables high-quality streaming of media content over the Internet delivered from conventional HTTP web servers. Similar to Apple’s HTTP Live Streaming (HLS) solution, MPEG-DASH works by breaking the content into a sequence of small segments, which are served over HTTP. Each piece contains a short interval of playback time of content that is potentially many hours in duration, such as a movie or the live broadcast of a sports event.

MPEG DASH supports a Common Encryption mode (CENC), which Bento4 implements. Encrypted MPEG DASH presentations should also include the proper signaling in the MPD to inform the player of what DRM(s) can be used to obtain the decryption keys for the streams. An MPD can contain DRM signaling for several DRMs (either just one or multiple entries if the same stream can reach players with different DRM technologies).

Again MPEG Dash is based on HTTP, aka TCP. In that case, DRM encryption is usually based on a public, private key encryption scheme.

On the diagram, you can see a standard AVI container. The video data objects are x264/h264 frames, which most of the streaming protocols encrypt, encode, and stream.

Our Modified Streaming Protocol

As you can see from the upper paragraphs, every standard encryption protocol was designed to stream data from a centralized server to a list of devices. Most of them use the traditional HTTP delivery networks to speed up their streaming. In our case, we had an entirely different problem. We had to stream encrypted content from multiple body cameras to a centralized server and, after that, restream the video from the server to a web browser-based dashboard. LTE networks can be quite fast when you have proper coverage, but when your signal drops, your network speed drops significantly, as well. So we decided to design our video streaming protocol, and I shall list our requirements:

  • Based on UDP: Sending TCP data through LTE can hurt your performance a lot. That’s the reason we decided to establish our protocol on UDP and to implement packet control.
  • Based on X264: X264 is an open-source implementation of the H.264 protocol. It is already implemented in most Android devices and is supported natively. The encoding rate is reasonable.
  • Codec agnostic: In the future, we wanted to support H.265 and its open-source implementation. Thus the protocol had to be code agnostic.
  • To use hybrid encryption: Most of the listed protocols do not use a hybrid encryption approach. We wanted our protocol to have better authentication and encryption mechanism, and that’s why we decided to use hybrid-based encryption on top of RSA and AES-GCM. We changed the keyphrase and IV for AES on every packet frame sent to implement the encryption correctly.
  • Binary-based: Keeping in mind that LTE is usually sold using monthly plans. These plans are generally only a couple of gigabytes. So we ended up making a binary-based protocol. Any other protocols, and especially the semantic-based ones, would result in more significant data consumption.
  • Adaptive Bitrate: The LTE network bandwidth depends on how strong a radio signal your device has. The weaker the signal, the lower the bandwidth. We had to implement an adaptive bitrate strategy, which lowered the resolution in a weaker signal. This way, you could receive frames no matter how strong is your LTE cell signal.

Our proof of concept implementation managed to fulfill these requirements. The finished network protocol was fast enough and binary compatible. It supported adaptive bitrate and was code agnostic. 

On the diagram, you can see a sample datagram of this protocol. The MTU was 1500 bytes to support all kinds of equipment, but not only with jumbo frames.

We used an UUIDv4 and RSA signature for authentication purposes. After that, you have multiple fields as a counter in the index, date, packet size, and an array of bytes. The implementation stripped down an h.264 frame to multiple UDP packets and sent them together. The server combined them back to the h.264 packet and appended them to corresponding files. 

We saw that it is better to have adaptive logic on the codec level during our tests for the protocol. For example, a simple JPEG stream was much better when the signal was weaker.

In the next part, we shall discuss how we created our body camera device and its software. We shall discuss our streaming server implementation in the final third part, give you a budget, and explain why the whole business model did not work as expected.

Next part is here