Skip to content

Commit 0f483f8

Browse files
ksaricksaric
authored
[TSD-86] Create visual architecture of the project (#38)
* [TSD-86] Create visual architecture of the project. * [TSD-86] Fix tests.
1 parent cdf584a commit 0f483f8

File tree

10 files changed

+207
-155
lines changed

10 files changed

+207
-155
lines changed

‎README.md‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,48 @@ Please read the instructions available in [INSTALL.md](INSTALL.md)
2323
![system architecture](https://user-images.githubusercontent.com/15665039/40042756-5d92611e-585d-11e8-80b4-72677c451dd1.png)<br/>
2424
This is a use case diagram. Use case diagrams overview the usage requirements for a system. They are useful for presentations to management and/or project stakeholders, but for actual development, you will find that use cases provide significantly more value because they describe "the meat" of the actual requirements. For more details, please see [here](http://www.agilemodeling.com/artifacts/useCaseDiagram.htm)
2525

26+
## Architecture
27+
28+
The general architecture can be seen here. It's still missing some things, but it's sufficient for most of the test we currently have.
29+
30+
![log classifier architecture](https://user-images.githubusercontent.com/6264437/43259505-bdaedbf2-90d6-11e8-9b24-fbc6226a3a7e.png)
31+
32+
### Layers
33+
34+
A layer is a list of functions grouped together. For example, a database has functions for interacting with - the database! Yaay. The problem is that in most of the cases we need to be able to stub those functions - if we want to test some of the functions that _depend on the database_, then we have no option but to make them something that we can replace in runtime. And to replace them in runtime, we place them in a "layer", a record-of-functions that allows us to replace them easily. Like:
35+
```
36+
data IOLayer m = IOLayer
37+
{ iolAppendFile :: FilePath -> Text -> m ()
38+
, iolPrintText :: Text -> m ()
39+
, iolReadFile :: FilePath -> m Text
40+
, iolLogDebug :: Text -> m ()
41+
, iolLogInfo :: Text -> m ()
42+
}
43+
```
44+
45+
We can think of a layer like a changeable module system - we export the functions in the data structure, but we can change them in runtime.
46+
47+
### DataSource
48+
49+
The data source is the place we get our data from. Currently, it's fixated on Zendesk. This is an abstraction towards any form of persistant data. We could say that this is a layer, but it actually contains quite a bit more.
50+
Currently, the Zendesk types themselves are a pretty big part of the `DataSource` module.
51+
52+
### DataLayer
53+
54+
So the data layer is the abstraction layer which can currently be specialized to the @HTTPLayer@ or @DBLayer@. It makes sense to abstract this away, since this is the group of functions that interact with any data in the system.
55+
56+
#### HTTPLayer
57+
58+
Obviously, the direct way we can fetch data is using HTTP JSON requests on the Zendesk API (https://developer.zendesk.com/rest_api/docs/core/introduction). This layer is the layer responsible for that. It contains common functions that allows us to communicate with the Zendesk REST API.
59+
60+
#### HTTPNetworkLayer
61+
62+
This layer is the layer responsible for the low level HTTP communication. The @HTTPLayer@ is the layer that communicates with the Zendesk REST API using this layer.
63+
64+
#### DBLayer
65+
66+
This layer is responsible for caching the results that come from the @HTTPLayer@ and then allows us to fetch data from the database rather then the HTTP REST api which has request limits and takes much longer.
67+
2668
### Overview
2769

2870
- Many of the Daedalus's issues can be identified by analyzing the log file. The classifier will utilize this by analyzing the log file and map with possible solution and problem which can be provided to the end user.

‎log-classifier.cabal‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,9 @@ library
2525
LogAnalysis.Types
2626
Regex
2727
Statistics
28-
Util
2928
DataSource
29+
Configuration
30+
Util
3031
other-modules: Paths_log_classifier
3132
DataSource.DB
3233
DataSource.Types

‎src/Configuration.hs‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
-- Here we need to export just the public info.
2+
module Configuration
3+
( defaultConfig
4+
, basicIOLayer
5+
) where
6+
7+
import Universum
8+
9+
import DataSource.DB
10+
import DataSource.Http
11+
import DataSource.Types
12+
import HttpLayer (basicHTTPNetworkLayer)
13+
14+
15+
-- | The default configuration.
16+
defaultConfig :: Config
17+
defaultConfig = Config
18+
{ cfgAgentId = 0
19+
, cfgZendesk = "https://iohk.zendesk.com"
20+
, cfgToken = ""
21+
, cfgEmail = "[email protected]"
22+
, cfgAssignTo = 0
23+
, cfgKnowledgebase = []
24+
, cfgNumOfLogsToAnalyze = 5
25+
, cfgIsCommentPublic = False -- TODO(ks): For now, we need this in CLI.
26+
, cfgDataLayer = basicDataLayer
27+
, cfgHTTPNetworkLayer = basicHTTPNetworkLayer
28+
, cfgIOLayer = basicIOLayer
29+
, cfgDBLayer = connDBLayer
30+
}
31+
32+
-- | The @IO@ layer.
33+
basicIOLayer :: (MonadIO m, MonadReader Config m) => IOLayer m
34+
basicIOLayer = IOLayer
35+
{ iolAppendFile = appendFile
36+
, iolPrintText = putTextLn -- iolPrintConsole
37+
, iolReadFile = readFile
38+
, iolLogDebug = putTextLn
39+
, iolLogInfo = putTextLn
40+
}
41+

‎src/DataSource.hs‎

Lines changed: 0 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -3,42 +3,8 @@ module DataSource
33
( module DataSource.Types
44
, module DataSource.Http
55
, module DataSource.DB
6-
, defaultConfig
7-
, basicIOLayer
86
) where
97

10-
import Universum
11-
128
import DataSource.DB
139
import DataSource.Http
1410
import DataSource.Types
15-
import HttpLayer (basicHTTPNetworkLayer)
16-
17-
18-
-- | The default configuration.
19-
defaultConfig :: Config
20-
defaultConfig = Config
21-
{ cfgAgentId = 0
22-
, cfgZendesk = "https://iohk.zendesk.com"
23-
, cfgToken = ""
24-
, cfgEmail = "[email protected]"
25-
, cfgAssignTo = 0
26-
, cfgKnowledgebase = []
27-
, cfgNumOfLogsToAnalyze = 5
28-
, cfgIsCommentPublic = False -- TODO(ks): For now, we need this in CLI.
29-
, cfgZendeskLayer = basicZendeskLayer
30-
, cfgHTTPNetworkLayer = basicHTTPNetworkLayer
31-
, cfgIOLayer = basicIOLayer
32-
, cfgDBLayer = connDBLayer
33-
}
34-
35-
-- | The @IO@ layer.
36-
basicIOLayer :: (MonadIO m, MonadReader Config m) => IOLayer m
37-
basicIOLayer = IOLayer
38-
{ iolAppendFile = appendFile
39-
, iolPrintText = putTextLn -- iolPrintConsole
40-
, iolReadFile = readFile
41-
, iolLogDebug = putTextLn
42-
, iolLogInfo = putTextLn
43-
}
44-

‎src/DataSource/DB.hs‎

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ module DataSource.DB
1111
-- * Empty layer
1212
, emptyDBLayer
1313
-- * Single connection
14-
, connZendeskLayer
15-
, connPoolZendeskLayer
14+
, connDataLayer
15+
, connPoolDataLayer
1616
-- * Connection pool
1717
, connDBLayer
1818
, connPoolDBLayer
@@ -34,13 +34,13 @@ import Database.SQLite.Simple.Internal (Connection, Field (..))
3434
import Database.SQLite.Simple.Ok (Ok (..))
3535
import Database.SQLite.Simple.ToField (ToField (..))
3636

37-
import DataSource.Http (basicZendeskLayer)
37+
import DataSource.Http (basicDataLayer)
3838
import DataSource.Types (Attachment (..), AttachmentContent (..), AttachmentId (..),
3939
Comment (..), CommentBody (..), CommentId (..), Config,
4040
DBLayer (..), TicketField (..), TicketFieldId (..),
4141
TicketFieldValue (..), TicketId (..), TicketInfo (..),
4242
TicketStatus (..), TicketTags (..), TicketURL (..), UserId (..),
43-
ZendeskLayer (..))
43+
DataLayer (..))
4444

4545
------------------------------------------------------------
4646
-- Single connection, simple
@@ -113,18 +113,18 @@ emptyDBLayer = DBLayer
113113

114114
-- | The simple connection Zendesk layer. Used for database querying.
115115
-- We need to sync occasionaly.
116-
connZendeskLayer :: forall m. (MonadIO m, MonadReader Config m) => ZendeskLayer m
117-
connZendeskLayer = ZendeskLayer
116+
connDataLayer :: forall m. (MonadIO m, MonadReader Config m) => DataLayer m
117+
connDataLayer = DataLayer
118118
{ zlGetTicketInfo = \tId -> withProdDatabase $ \conn -> getTicketInfoByTicketId conn tId
119-
, zlListDeletedTickets = zlListDeletedTickets basicZendeskLayer
119+
, zlListDeletedTickets = zlListDeletedTickets basicDataLayer
120120
, zlListAssignedTickets = \uId -> withProdDatabase $ \conn -> getAllAssignedTicketsByUser conn uId
121121
, zlListRequestedTickets = \uId -> withProdDatabase $ \conn -> getAllRequestedTicketsByUser conn uId
122122
, zlListUnassignedTickets = withProdDatabase getAllUnassignedTicketsByUser
123-
, zlListAdminAgents = zlListAdminAgents basicZendeskLayer
123+
, zlListAdminAgents = zlListAdminAgents basicDataLayer
124124
, zlGetAttachment = \att -> withProdDatabase $ \conn -> DataSource.DB.getAttachmentContent conn att
125125
, zlGetTicketComments = \tId -> withProdDatabase $ \conn -> getTicketComments conn tId
126-
, zlPostTicketComment = zlPostTicketComment basicZendeskLayer
127-
, zlExportTickets = zlExportTickets basicZendeskLayer
126+
, zlPostTicketComment = zlPostTicketComment basicDataLayer
127+
, zlExportTickets = zlExportTickets basicDataLayer
128128
}
129129

130130

@@ -147,18 +147,18 @@ connDBLayer = DBLayer
147147

148148
-- | The connection pooled Zendesk layer. Used for database querying.
149149
-- We need to sync occasionaly.
150-
connPoolZendeskLayer :: forall m. (MonadBaseControl IO m, MonadIO m, MonadReader Config m) => DBConnPool -> ZendeskLayer m
151-
connPoolZendeskLayer connPool = ZendeskLayer
150+
connPoolDataLayer :: forall m. (MonadBaseControl IO m, MonadIO m, MonadReader Config m) => DBConnPool -> DataLayer m
151+
connPoolDataLayer connPool = DataLayer
152152
{ zlGetTicketInfo = \tId -> withConnPool connPool $ \conn -> getTicketInfoByTicketId conn tId
153-
, zlListDeletedTickets = zlListDeletedTickets basicZendeskLayer
153+
, zlListDeletedTickets = zlListDeletedTickets basicDataLayer
154154
, zlListAssignedTickets = \uId -> withConnPool connPool $ \conn -> getAllAssignedTicketsByUser conn uId
155155
, zlListRequestedTickets = \uId -> withConnPool connPool $ \conn -> getAllRequestedTicketsByUser conn uId
156156
, zlListUnassignedTickets = withConnPool connPool getAllUnassignedTicketsByUser
157-
, zlListAdminAgents = zlListAdminAgents basicZendeskLayer
157+
, zlListAdminAgents = zlListAdminAgents basicDataLayer
158158
, zlGetAttachment = \att -> withConnPool connPool $ \conn -> DataSource.DB.getAttachmentContent conn att
159159
, zlGetTicketComments = \tId -> withConnPool connPool $ \conn -> getTicketComments conn tId
160-
, zlPostTicketComment = zlPostTicketComment basicZendeskLayer
161-
, zlExportTickets = zlExportTickets basicZendeskLayer
160+
, zlPostTicketComment = zlPostTicketComment basicDataLayer
161+
, zlExportTickets = zlExportTickets basicDataLayer
162162
}
163163

164164

‎src/DataSource/Http.hs‎

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
{-# LANGUAGE ScopedTypeVariables #-}
44

55
module DataSource.Http
6-
( basicZendeskLayer
7-
, emptyZendeskLayer
6+
( basicDataLayer
7+
, emptyDataLayer
88
, createResponseTicket
99
) where
1010

@@ -25,7 +25,7 @@ import DataSource.Types (Attachment (..), AttachmentContent (..), Comm
2525
DeletedTicket (..), ExportFromTime (..), FromPageResultList (..),
2626
PageResultList (..), Ticket (..), TicketId (..), TicketInfo (..),
2727
TicketTag (..), TicketTags (..), User, UserId (..),
28-
ZendeskAPIUrl (..), ZendeskLayer (..), ZendeskResponse (..),
28+
ZendeskAPIUrl (..), DataLayer (..), ZendeskResponse (..),
2929
asksHTTPNetworkLayer, parseComments, renderTicketStatus, showURL)
3030

3131
-- ./mitmproxy --mode reverse:https://iohk.zendesk.com -p 4001
@@ -39,8 +39,8 @@ import DataSource.Types (Attachment (..), AttachmentContent (..), Comm
3939
-- - get returns a single result (wrapped in @Maybe@)
4040
-- - list returns multiple results
4141
-- - post submits a result (maybe PUT?!)
42-
basicZendeskLayer :: (MonadIO m, MonadReader Config m) => ZendeskLayer m
43-
basicZendeskLayer = ZendeskLayer
42+
basicDataLayer :: (MonadIO m, MonadReader Config m) => DataLayer m
43+
basicDataLayer = DataLayer
4444
{ zlGetTicketInfo = getTicketInfo
4545
, zlListDeletedTickets = listDeletedTickets
4646
, zlListRequestedTickets = listRequestedTickets
@@ -54,8 +54,8 @@ basicZendeskLayer = ZendeskLayer
5454
}
5555

5656
-- | The non-implemented Zendesk layer.
57-
emptyZendeskLayer :: forall m. (Monad m) => ZendeskLayer m
58-
emptyZendeskLayer = ZendeskLayer
57+
emptyDataLayer :: forall m. (Monad m) => DataLayer m
58+
emptyDataLayer = DataLayer
5959
{ zlGetTicketInfo = \_ -> error "Not implemented zlGetTicketInfo!"
6060
, zlListDeletedTickets = pure []
6161
, zlListRequestedTickets = \_ -> error "Not implemented zlListRequestedTickets!"

‎src/DataSource/Types.hs‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -40,14 +40,14 @@ module DataSource.Types
4040
-- * General configuration
4141
, App
4242
, Config (..)
43-
, ZendeskLayer (..)
43+
, DataLayer (..)
4444
, HTTPNetworkLayer (..)
4545
, IOLayer (..)
4646
, DBLayer (..)
4747
, knowledgebasePath
4848
, tokenPath
4949
, assignToPath
50-
, asksZendeskLayer
50+
, asksDataLayer
5151
, asksHTTPNetworkLayer
5252
, asksIOLayer
5353
, asksDBLayer
@@ -115,7 +115,7 @@ data Config = Config
115115
-- ^ Number of files classifier will analyze
116116
, cfgIsCommentPublic :: !Bool
117117
-- ^ If the comment is public or not, for a test run we use an internal comment.
118-
, cfgZendeskLayer :: !(ZendeskLayer App)
118+
, cfgDataLayer :: !(DataLayer App)
119119
-- ^ The Zendesk API layer. We will ideally move this into a
120120
-- separate configuration containing all the layer (yes, there a couple of them).
121121
, cfgIOLayer :: !(IOLayer App)
@@ -129,14 +129,14 @@ data Config = Config
129129
}
130130

131131

132-
-- | Utility function for getting a function of the @ZendeskLayer@.
133-
asksZendeskLayer
132+
-- | Utility function for getting a function of the @DataLayer@.
133+
asksDataLayer
134134
:: forall m a. (MonadReader Config m)
135-
=> (ZendeskLayer App -> a)
135+
=> (DataLayer App -> a)
136136
-> m a
137-
asksZendeskLayer getter = do
137+
asksDataLayer getter = do
138138
Config{..} <- ask
139-
pure $ getter cfgZendeskLayer
139+
pure $ getter cfgDataLayer
140140

141141
-- | Utility function for getting a function of the @cfgHTTPNetworkLayer@.
142142
asksHTTPNetworkLayer
@@ -183,7 +183,7 @@ assignToPath = "./tmp-secrets/assign_to"
183183
-- | The Zendesk API interface that we want to expose.
184184
-- We don't want anything to leak out, so we expose only the most relevant information,
185185
-- anything relating to how it internaly works should NOT be exposed.
186-
data ZendeskLayer m = ZendeskLayer
186+
data DataLayer m = DataLayer
187187
{ zlGetTicketInfo :: TicketId -> m (Maybe TicketInfo)
188188
, zlListDeletedTickets :: m [DeletedTicket]
189189
, zlListRequestedTickets :: UserId -> m [TicketInfo]

0 commit comments

Comments
 (0)