SQL is simpler to understand for majority of users. It's easier to get started without learning a tool chain, programming best practices, etc. that could present a challenge to new users. SQL is also great at representing relationships between datasets and developing business logic transformations tends to be simpler and easier to understand. Oftentimes, when using another programming language you end up with many modules, imported libraries, code hacks and optimizations. It all can very quickly make it difficult to read.

One example is dbt. They started as writing simple SQL models. With the introduction of Jinja, majority of models look nothing like SQL anymore. You need to visualize your model to understand relationships. It took the beauty of SQL and mucked it.

At Upsolver we built a streaming+batch ETL tool that lets you build data pipelines in SQL. We did it because it's easier for non-data engineers to get started, easy to version and maintain as code and easy to automate (not that you can't do this with other languages). The same goes with kSQLDB, Materialize and even Spark and Flink use SQL as a way to simplify onboarding for non-developers.

> One example is dbt. They started as writing simple SQL models. With the introduction of Jinja, majority of models look nothing like SQL anymore. You need to visualize your model to understand relationships. It took the beauty of SQL and mucked it.

I know you are pitching your startup, but that's just completely untrue.

The dbt site describes this as a simple example:

    select * from {{ref('really_big_table')}}
    
    {% if incremental and target.schema == 'prod' %}
      where timestamp >= (select max(timestamp) from {{this}})
     {% else %}
      where timestamp >= dateadd(day, -3, current_date)
    {% endif %}

As someone who has written 100s (maybe 1000s at this point) of DBT models, the amount of Jinja you need is at most 1-2% of the codebase.

looks pretty impressive. the fact they do manipulation in JS and don't touch markup or CSS is a great thing

it's a great write up and the context issue seems interesting, however it's from Feb 2010 according to the last paragraph and a lot has changed in V8 since

Thank you this is a really great starting point.

I don't see the links you mentioned but I came across this one the other day: http://pubsub.io/

You can also build one using Socket.IO

Unfortunatley the links did not get pasted .

I was talking about https://github.com/jazzychad/fanout.node.js http://blog.redistogo.com/2010/08/18/node-and-redis-pub-sub/

I'm working on a social networking service and would be interested to see if you can help. It's still very much under development but check it out to get a feel. familyengage.com

I gave a presentation to in Japan (I don't speak Japanese) to about 200 people through a interpreter. The challenge there is that you must be able to speak slowly, clearly and use non-complicated and less technical words. Since my presentation was technical it was difficult to stay away from technical terms and I later found out that the interpreters did a poor job translating my presentation and many people didn't really get a lot of the detail I discussed. My suggestion is to keep things simple, find interesting ways to explain your technology,product,ideas,etc in a non-technical way (as you were explaining it to your mother). This is obvious, but it becomes that much more important when your presenting to an international audience especially when you are going through an interpreter.

Roy

very true but with the scale of users and the rate of growth the opportunities are also very large. AS for dealing with studios and such, usually that's not the startup's problem, that's something the operators will work through if your company is bringing real value. You are right that's not a fun process but it's quickly changing since the operators are losing money to the Internet (if you may) and realize things need to change.

I'm involved in some very interesting projects where this view of operators as being slow and difficult is changing.

1. I make sure the job description is inline with what I want to do. I also make sure it's documented in case 2 months into your new job your boss decides you need to focus on something else and you end up pigeon holed into something you don't want to do. 2. Make sure there is a solid career path that could help you grow within the organization. Ask the employer to highlight such a path for you. 3. Compensation, benefits, stock options, bonuses, etc. Make sure this is all clearly documented in the offer letter. If they say no to documenting it they will most likely end up not giving it to you once you are on board. 4. Make sure you clearly understand the company's strategy, growth plan and financials. You don't want to get into a place that will end up on it face a year later. Many times it's easy to spot such companies with simple research. If their strategy doesn't agree with you, you will end up frustrated very quickly. 5. I usually also do research on the products being sold and try to understand how I can personally add value to make these products more successful. This helps me determine if I will enjoy working on them or if I would be bored. Companies strategy might be good but if the current products are blah you'll be bored quickly and might not have the patience to wait for the strategy to unfold.

You should use an extra line break between paragraphs in order to avoid wall-of-text mode.