Apache Airflow (Google Compose) and Apache Beam (Google DataFlow)

 Airflow is a task management system. The nodes of the DAG [directed acyclic graph (DAG)] are tasks and Airflow makes sure to run them in the proper order, making sure one task only starts once its dependency tasks have finished. Dependent tasks don't run at the same time but only one after another. Independent tasks can run concurrently.

Airflow can do anything. It has BashOperator and PythonOperator which means it can run any bash script or any Python script.

It is a way to organize (setup complicated data pipeline DAGs), schedule, monitor, trigger re-runs of data pipelines, in a easy-to-view and use UI.

Also, it is easy to setup and everything is in familiar Python code.

Airflow manages tasks, which depend on one another. While this dependency can consist of one task passing data to the next one, that is not a requirement. In fact Airflow doesn't even care what the tasks do, it just needs to start them and see if they finished or failed. If tasks need to pass data to one another you need to co-ordinate that yourself, telling each task where to read and write its data, e.g. a local file path or a web service somewhere. Tasks can consist of Python code but they can also be any external program or a web service call.

when using google cloud, we may need a folder to save temp data from one step to another, or save custom python script to call it through DAG

Here is the available paths 

here is a sample DAG file

https://github.com/GoogleCloudPlatform/python-docs-samples/blob/c5635d1146fc2c0ff284c41d4b2d1132b25ae270/composer/workflows/simple.py

Apache Beam is a wrapper for the many data processing frameworks (Spark, Flink etc.) out there.

The intent is so you just learn Beam and can run on multiple backends (Beam runners).

If you are familiar with Keras and TensorFlow/Theano/Torch, the relationship between Keras and its backends is similar to the relationship between Beam and its data processing backends.

Beam is a dataflow engine. The nodes of the DAG form a (possibly branching) pipeline. All the nodes in the DAG are active at the same time, and they pass data elements from one to the next, each doing some processing on it.

In Beam, your step definitions are tightly integrated with the engine. You define the steps in a supported programming language and they run inside a Beam process. Handling the computation in an external process would be difficult. Your steps only need to worry about the computation they're performing, not about storing or transferring the data. Transferring the data between different steps is handled entirely by the framework.

Beam is actually an abstraction layer. Beam pipelines can run on Apache Spark, Apache Flink, Google Cloud Dataflow and others. All of these support a more or less similar programming model. 

Flutter Filter Array

List<String> SearchResult = CitiesService.getSuggestions(SearchText);

......

class CitiesService {
  static final List<String> cities = [
    'Beirut',
    'Damascus',
    'San Fransisco',
  ];

  static List<String> getSuggestions(String query) {
    List<String> matches = <String>[];
    matches.addAll(cities);

    matches.retainWhere((s) => s.toLowerCase().contains(query.toLowerCase()));
    return matches;
  }
}

flutter web

How to call external java script from flutter method?

Step 1: Include javascript in Project\Web\Index.html

Step 2: import 'dart:js' as js;

Step 3:  js.context.callMethod('__javascript function name___', ['function parameters']);

How to listen to javascript event and pass data to flutter?

you can post a message in your js code when the event occurs

window.parent.postMessage('any message', "*");

and listen to it in your dart code

@override
void initState() {
  super.initState();

  //setup listener ---------------------------------
  window.addEventListener("message", (event) {
    MessageEvent event2 = event;
    print(event2.data);
  });
  //------------------------------------------------

}

How to include custom Div with ID in flutter?

Step 1: 

import 'dart:html';
import 'dart:ui' as ui;

Step 2: Call the next widget with expected Div ID

class CustomDiv extends StatelessWidget {
  String DivName;
   CustomDiv({Key key,String this.DivName }) : super(key: key);

  @override
  Widget build(BuildContext context) {
    final _element = DivElement()
      ..id = DivName
      ..innerHtml = 'Hello World from a Div'
      ..style.width = '400'
      ..style.height = '200'
      ..style.backgroundColor='#f7f7f7'
      ..style.border = '1';

    ///First create a div and register it
    // ignore: undefined_prefixed_name
    ui.platformViewRegistry.registerViewFactory(DivName, (int viewId) => _element);

    return SizedBox(
        width: 400,
        height: 200,
        child: HtmlElementView( viewType: DivName, ));
  }
}

How to include normal HTML inside flutter?

use EasyWebView widget

Step 1: in dependencies add easy_web_view: 

Step 2: import 'package:easy_web_view/easy_web_view.dart';

Step 3: Include the widget

class WebViewExample1 extends StatelessWidget {
   WebViewExample1({Key key}) : super(key: key);

  final ValueKey key1 = ValueKey('key_0');

  @override
  Widget build(BuildContext context) {

    return  EasyWebView(
      width: 340,
      height: 220,
      src: htmlContent,
      onLoaded: () {
        print('Loaded...');
        //js.context.callMethod('enableVideo', ['Flutter is calling upon JavaScript!']);

      },
      isHtml: true,
      isMarkdown: false,
      convertToWidgets: false,
      key: key1,
      widgetsTextSelectable: true,
      webNavigationDelegate: (_) =>   WebNavigationDecision.navigate,
      crossWindowEvents: [
        CrossWindowEvent(
            name: 'Test',
            eventAction: (eventMessage) {
              print('Event message: $eventMessage');
            }),
      ],
      // width: 100,
      // height: 100,
    );
  }

  String get htmlContent => """
<!DOCTYPE html>
<html>
<head>


</head>
<body bgcolor="#ff0000">
<h1>Hiiiiiiiiiiiiiii</h1>
</body>
</html>
""";

}